Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large scale network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57
All sourcesNASA NTRSDOE OSTI
NASA NTRS1991

Video data compression using artificial neural network differential vector quantization

An artificial neural network vector quantizer is developed for use in data compression applications such as Digital Video. Differential Vector Quantization is used to preserve edge features, and a new adaptive algorithm, known as Frequency-Sensitive Competitive Learning, is used to develop the vector quantizer codebook. To develop real time performance, a custom Very Large Scale Integration Application Specific Integrated Circuit (VLSI ASIC) is being developed to realize the associative memory functions needed in the vector quantization algorithm. By using vector quantization, the need for Huffman coding can be eliminated, resulting in superior performance against channel bit errors than methods that use variable length codes.

Krishnamurthy, Ashok K.↗
NASA NTRS2001

Latency Hiding in Dynamic Partitioning and Load Balancing of Grid Computing Applications

The Information Power Grid (IPG) concept developed by NASA is aimed to provide a metacomputing platform for large-scale distributed computations, by hiding the intricacies of highly heterogeneous environment and yet maintaining adequate security. In this paper, we propose a latency-tolerant partitioning scheme that dynamically balances processor workloads on the.IPG, and minimizes data movement and runtime communication. By simulating an unsteady adaptive mesh application on a wide area network, we study the performance of our load balancer under the Globus environment. The number of IPG nodes, the number of processors per node, and the interconnected speeds are parameterized to derive conditions under which the IPG would be suitable for parallel distributed processing of such applications. Experimental results demonstrate that effective solution are achieved when the IPG nodes are connected by a high-speed asynchronous interconnection network.

Das, Sajal K.↗
NASA NTRS2005

Mapping of ice layer extent and snow accumulation in the percolation zone of the Greenland ice sheet

The Greenland ice sheet underwent record extensive melt in 2002 and prolonged melt in 2003. The severe melting created a significant and extensive ice layer over the Greenland ice sheet. An innovative approach is developed to detect the ice layer formation using data acquired by the SeaWinds scatterometer on the QuikSCAT satellite. QuikSCAT backscatter together with in situ data from automatic weather stations of the Greenland Climate Network are used to map the extent of ice layer formation. The results reveal areas of extensive ice layer formed by the 2002 melt, which is consistent with the maximum melt extent in 2002. Moreover, during freezing seasons, QuikSCAT data show a linear decrease in backscatter (in decibels or dB) that is related to the amount of snow accumulation in the ice layer formation region. This snow accumulation signature is caused by the attenuation of radar waves in the snow layer, accumulating since the last major melt event, whose thickness appears as an exponential function in relation to the backscatter signature. We use the Greenland Climate Network data to calibrate the QuikSCAT accumulation rate in order to estimate and map snow accumulation. QuikSCAT results capture the extreme snowfall in mid-April 2003, which deposited more than 0.5 m of snow in a day as measured by the automated weather station at the NASA South East site. Large-scale QuikSCAT results show an anomalous increase of snow accumulation over the southeast region of Greenland during the 2002-2003 freezing season.

snow↗
DOE OSTI2022

Machine Learning for Improving Surface-Layer-Flux Estimates

Abstract Flows in the atmospheric boundary layer are turbulent, characterized by a large Reynolds number, the existence of a roughness sublayer and the absence of a well-defined viscous layer. Exchanges with the surface are therefore dominated by turbulent fluxes. In numerical models for atmospheric flows, turbulent fluxes must be specified at the surface; however, surface fluxes are not known a priori and therefore must be parametrized. Atmospheric flow models, including global circulation, limited area models, and large-eddy simulation, employ Monin–Obukhov similarity theory (MOST) to parametrize surface fluxes. The MOST approach is a semi-empirical formulation that accounts for atmospheric stability effects through universal stability functions. The stability functions are determined based on limited observations using simple regression as a function of the non-dimensional stability parameter representing a ratio of distance from the surface and the Obukhov length scale (Obukhov in Trudy Inst Theor Geofiz AN SSSR 1:95–115, 1946), $$z/L$$ z / L . However, simple regression cannot capture the relationship between governing parameters and surface-layer structure under the wide range of conditions to which MOST is commonly applied. We therefore develop, train, and test two machine-learning models, an artificial neural network (ANN) and random forest (RF), to estimate surface fluxes of momentum, sensible heat, and moisture based on surface and near-surface observations. To train and test these machine-learning algorithms, we use several years of observations from the Cabauw mast in the Netherlands and from the National Oceanic and Atmospheric Administration’s Field Research Division tower in Idaho. The RF and ANN models outperform MOST. Even when we train the RF and ANN on one set of data and apply them to the second set, they provide more accurate estimates of all of the fluxes compared to MOST. Estimates of sensible heat and moisture fluxes are significantly improved, and model interpretability techniques highlight the logical physical relationships we expect in surface-layer processes.

Meteorology & Atmospheric Sciences↗
DOE OSTI2025

Real-Time event reconstruction for Nuclear Physics Experiments using Artificial Intelligence

Charged track reconstruction is a critical task in nuclear physics experiments, enabling the identification and analysis of particles produced in high-energy collisions. Machine learning (ML) has emerged as a powerful tool for this purpose, addressing the challenges posed by complex detector geometries, high event multiplicities, and noisy data. Traditional methods rely on pattern recognition algorithms like the Kalman filter, but ML techniques, such as neural networks, graph neural networks (GNNs), and recurrent neural networks (RNNs), offer improved accuracy and scalability. By learning from simulated and real detector data, ML models can identify and classify tracks, predict trajectories, and handle ambiguities caused by overlapping or missing hits. Moreover, ML-based approaches can process data in near-real-time, enhancing the efficiency of experiments at large-scale facilities like the Large Hadron Collider (LHC) and Jefferson Lab (JLAB). As detector technologies and computational resources evolve, ML-driven charged track reconstruction continues to push the boundaries of precision and discovery in nuclear physics. In these proceedings, we highlight advancements in charged track identification leveraging Artificial Intelligence within the CLAS12 detector, achieving a notable enhancement in experimental statistics compared to traditional methods. Additionally, we showcase real-time event reconstruction capabilities, including the inference of charged particle properties, such as momentum, direction, and species identification, at speeds matching data acquisition rates. These innovations enable the extraction of physics observables directly from the experiment in real-time.

Gavalian, Gagik (ORCID:0000000267385457)↗
DOE OSTI2025

Recent Experience with the CMS Data Management System

The CMS[1] experiment manages a large-scale data infrastructure, currently handling over 200 PB of disk and 500 PB of tape storage and transferring more than 1 PB of data per day on average between various WLCG[2] sites. Utilizing Rucio[3] for high-level data management, FTS[4] for data transfers, and a variety of storage and network technologies at the sites, CMS confronts inevitable challenges due to the system’s growing scale and evolving nature. Key challenges include managing transfer and storage failures, optimizing data distribution across different storages based on production and analysis needs, implementing necessary technology upgrades and migrations, and efficiently handling user requests. The data management team has established comprehensive monitoring to supervise this system and has successfully addressed many of these challenges. The team’s efforts aim to ensure data availability and protection, minimize failures and manual interventions, maximize transfer throughput and resource utilization, and provide reliable user support. This paper details the operational experience of CMS with its data management system in recent years, focusing on the encountered challenges, the effective strategies employed to overcome them and the ongoing challenges as we prepare for future demands.

Öztürk, Hasan [CERN]↗
DOE OSTI2022

WISE-PS1-STRM: neural network source classification and photometric redshifts for WISE×PS1

ABSTRACT We cross-match between the WISE All-Sky and PS1 3π DR2 source catalogues. The resulting catalogue has 354 590 570 objects, significantly fewer than the parent PS1 catalogue, but its combination of optical and infrared colours facilitate both better source classification and photometric redshift estimation. We perform a neural network-based classification of the objects into galaxies, quasars, and stars, then run neural network-based photometric redshift estimation for the galaxies. The star sample purity and quasar sample completeness measures improve substantially, and the resulting photo-z’s are significantly more accurate in terms of statistical scatter and bias than those calculated from PS1 properties alone. The catalogue will be a basis for future large-scale structure studies, and will be made available as a high-level science product via the Mikulski Archive for Space Telescopes.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

A Privacy Preserving Model-Free Optimization and Control Framework for Demand Response from Residential Thermal Loads

We consider the problem of optimizing the cost of procuring electricity for a large collection of homes managed by a load serving entity, by pre-cooling or pre-heating the thermal inertial loads in the homes to avoid procuring power during periods of peak electricity pricing. We would like to accomplish this objective in a completely privacy-preserving and model-free manner, that is, without direct access to the state variables (temperatures or power consumption) or the dynamical models (thermal characteristics) of individual homes, while guaranteeing personal comfort constraints of the consumers. We propose a two-stage optimization and control framework to address this problem. In the first stage, we use a long short-term memory (LSTM) network to predict hourly electricity prices, based on historical pricing data and weather forecasts. Given the hourly price forecast and thermal models of the homes, the problem of designing an optimal power consumption trajectory that minimizes the total electricity procurement cost for the collection of thermal loads can be formulated as a large-scale integer program (with millions of variables) due to the on-off cyclical dynamics of such loads. We provide a simple heuristic relaxation to make this large-scale optimization problem model-free and computationally tractable. In the second stage, we translate the results of this optimization problem into distributed open-loop control laws that can be implemented at individual homes without measuring or estimating their state variables, while simultaneously ensuring consumer comfort constraints. We demonstrate the performance of this approach on a large-scale test case comprising of 500 homes in the Houston area and benchmark its performance against a direct model-based optimization and control solution.

Sivaranjani, S.↗
DOE OSTI2026

Automated, reliable, and efficient continental-scale replication of 7.3 petabytes of computational simulation data: A case study

We report on our experiences replicating 7.3 petabytes (PB) of Earth System Grid Federation (ESGF) computational simulation data from Lawrence Livermore National Laboratory (LLNL) in California to Argonne National Laboratory (ANL) in Illinois and Oak Ridge National Laboratory (ORNL) in Tennessee—a task motivated by a need for increased reliability, capacity, and performance. This task presented significant challenges: the need to move 29 million files twice under time pressure from aging storage hardware; a source file system bottleneck limiting throughput to 1.5 GB/s; frequent site maintenance windows; and the need for complete reliability at scale. We addressed these challenges using a simple replication tool that invoked Globus to transfer large bundles of files while tracking progress in a database, dynamically rerouting transfers to work around maintenance periods and file system limitations. Under the covers, Globus organized transfers to make efficient use of the high-speed Energy Sciences network (ESnet) and the data transfer nodes deployed at participating sites, and also addressed security, integrity checking, and recovery from a variety of transient failures. This success demonstrates the considerable benefits that can accrue from the adoption of performant data replication infrastructure. The replication tool is available at https://github.com/esgf2-us/data-replication-tools.

Globus↗
DOE OSTI2022

Physics-informed machine learning modeling for predictive control using noisy data

Due to the occurrence of over-fitting at the learning phase, the modeling of chemical processes via artificial neural networks (ANN) by using corrupted data (i.e., noisy data) is an ongoing challenge. Therefore, this work investigates the effect of both Gaussian and non-Gaussian noise on the performance of process-structure based recurrent neural networks (RNN) models, which take the form of partially-connected RNN models in this work, that are used to approximate a class of multi-input-multi-outputs nonlinear systems. Furthermore, two different techniques, specifically Monte Carlo dropout and co-teaching, are utilized in the development of partially-connected RNN models. Here, these two techniques are employed to reduce the over-fitting in ANNs when noisy data is used in the training process and, hence, to improve the open-loop accuracy as well as the closed-loop performance under a Lyapunov-based model predictive controller (MPC). Aspen Plus Dynamics, a well-known high-fidelity process simulator, is used to simulate a large-scale chemical process application in order to demonstrate the anticipated improvements in both open-loop approximation and closed-loop controller performance in the presence of Gaussian and non-Gaussian noise in the data set using physics-informed RNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Integrating Analytical Solutions and U-Net Model for Predicting Groundwater Contaminant Plumes in Pump-and-Treat Systems

Pump-and-treat (P&T) is a common technique for groundwater remediation involving the extraction and treatment of contaminated water above ground. Optimizing the design and operation of the P&T well network is essential for maximizing the system’s effectiveness and efficiency. However, this optimization often necessitates many model evaluations, leading to computationally demanding tasks. This study introduces a novel approach that integrates analytical solutions for groundwater dynamics with the U-Net (Ronneberger et al., 2015) deep learning framework to predict groundwater contaminant plume migration under dynamic pumping conditions. By incorporating the Thiem equation (Thiem, 1906) into the input preprocessing, the U-Net model transforms sparse well data into a continuous spatial field that captures the hydraulic impacts of pumping activities. This integration enables the model to leverage both deep learning capabilities and classical physics-based groundwater theories, enhancing prediction accuracy and computational efficiency. These advancements can facilitate rapid, large-scale evaluations of P&T optimization simulations, allowing for timely and effective decision-making in well placement and system management. We demonstrate the model's robust performance across both simplified transient 2D models and a more complex 3D heterogeneous site model at the 200 West P&T facility at the Hanford Site. The U-Net-based model offers substantial computational advantages, reducing simulation times significantly compared to full physics-based models and providing a powerful tool for rapid site evaluation and P&T system optimization, such as evaluating alternative P&T well network designs. Our findings highlight the potential of advanced machine learning models to significantly enhance the efficiency and sustainability of groundwater remediation efforts, offering a novel application of U-Net architecture in environmental science.

Pump-and-treat↗
DOE OSTI2023

Energy Storage Sizing, Placement and Valuation for Distribution Networks

SAND2023-05682O The Energy Storage Sizing, Placement, and Valuation for Distribution software allows a user to simulate sizing, placement, and operation of one or more storage assets on a distribution grid. Among its features are measuring differences to determine the best combinations, and options for configuring and comparing results. This new simulation capability leverages the Open Distribution System Simulator and the Hierarchical Engine for Large-scale Infrastructure Co-Simulation (HELICS). Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Tamrakar, Ujjwol↗
DOE OSTI2023

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Quantitative relationships between film morphology, charge carrier dynamics, and photovoltaic performance in bulk-heterojunction binary vs. ternary acceptor blends

Addressing pertinent and perplexing questions regarding why nonfullerene acceptors (NFAs) promote higher power conversion efficiencies (PCEs) than traditional fullerenes and how photoactive bulk heterojunction (BHJ) film morphology, charge photogeneration, and recombination dynamics dictate solar cell performance have stimulated many studies of polymer solar cells (PSCs), yet quantitative relationships remain limited. Better understanding in these areas offers the potential to advance materials design and device engineering, afford higher PCEs, and ultimate commercialization. Here we probe quantitative relationships between BHJ film morphology, charge carrier dynamics, and photovoltaic performance in model binary and ternary blend systems having a wide bandgap donor polymer, a fullerene, and a promising NFA. We show that optimal PC 71 BM incorporation in a PBDB-TF:ITIC-Th binary system matrix retains the original π-face-on orientation, ITIC-Th crystallinity and BHJ film crystallite dimensions, and reduces film upper surface ITIC-Th segregation. Such morphology changes together simultaneously increase hole (μ h ) and electron (μ e ) mobilities, facilitate light-activated ITIC-Th to PC 71 BM domain electron delocalization, reduce free charge carrier (FC) bimolecular recombination (BR) within PBDB-TF:ITIC-Th mixed regions, and increase FC extraction pathways viaPBDB-TF:PC 71 BM pairs. The interplay of these effects yields significantly enhanced inverted cell short-circuit current density (J SC ), fill factor (FF), and PCE. Unexpectedly, we also find that excessive PC 71 BM in the PBDB-TF:ITIC-Th binary system alters the PBDB-TF orientation to π-edge-on, increases large scale PC 71 BM-rich aggregations and BHJ upper surface PC 71 BM composition. Further, these morphology changes increase parasitic decay processes such as intersystem crossing from photoexcited PC 71 BM, compromising the J SC , FF, and PCE metrics. ITIC-Th X-ray diffraction reveals a unique sidechain-dominated molecular network with previously unknown sidechain-end group stacking, rationalizing the STEM and GIWAXS results, photophysics, and the high μ e . DFT computation reveals charge transfer networks within ITIC-Th crystallites, supporting excited-state electron delocalization from ITIC-Th to PC 71 BM. This structure–property understanding leads to a newly reported NFA blend with PCE near 17%.

14 SOLAR ENERGY↗
DOE OSTI2020

Evolutionary Variation in MADS Box Dimerization Affects Floral Development and Protein Abundance in Maize

Interactions between MADS-box transcription factors are critical in the regulation of floral development, and shifting MADS-box protein-protein interactions are predicted to have influenced floral evolution. However, precisely how evolutionary variation in protein-protein interactions affects MADS-box protein function remains unknown. To assess the impact of changing MADS-box protein-protein interactions on transcription factor function, we turned to the grasses, where interactions between B-class MADS-box proteins vary. We tested the functional consequences of this evolutionary variability using maize as an experimental system. We found that differential B-class dimerization was associated with subtle, quantitative differences in stamen shape. In contrast, differential dimerization resulted in large-scale changes to downstream gene expression. Differential dimerization also affected B-class complex composition and abundance, independent of transcript levels. This indicates that differential B-class dimerization affects protein degradation, revealing an important consequence for evolutionary variability in MADS-box interactions. Our results highlight complexity in the evolution of developmental gene networks - changing protein-protein interactions could affect not only the composition of transcription factor complexes, but also their degradation and persistence in developing flowers. Here, our results also show how coding change in a pleiotropic master regulator could have small, quantitative effects on development.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Tribal Engagement in Transmission Planning

The purpose of this white paper is to provide Tribes with information that (a) helps assess whether transmission access is important for their energy goals, and (b) provides guidance on how to engage in transmission planning, if and when doing so would help the Tribe achieve its energy goals. DOE's Office of Indian Energy notes that "Indian Country contains vast untapped energy resources" (OIE 2023). Individual tribes express a wide array of energy needs and goals, ranging from simple access to basic electricity service and energy efficiency tools, to the development of utility-scale generating plants. Access to transmission is a key part of this multifaceted tribal energy picture. About 2.3 percent of the nation's transmission miles are on tribal lands (OIE 2023). Even within that slice, however, circumstances vary widely. Much of that transmission is for local network delivery (138kV or less). There are some large-capacity 500kV lines, but in many cases the Tribe has limited access to them even when they run across the reservation. Some of the Tribes with the largest land area have little transmission, and some such as Navajo Nation and Hopi Tribe have many homes with no access to electricity. This white paper begins with a high-level summary of transmission issues that can affect a Tribe's energy goals. The discussion includes findings from major transmission-related studies, including the 2023 National Transmission Needs Study (GDO 2023). It then discusses strategies that could help a Tribe safeguard its interests when those interests intersect transmission infrastructure planning.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS1993

Internetworking satellite and local exchange networks for personal communications applications

The demand for personal communications services has shown unprecedented growth, and the next decade and beyond promise an era in which the needs for ubiquitous, transparent and personalized access to information will continue to expand in both scale and scope. The exchange of personalized information is growing from two-way voice to include data communications, electronic messaging and information services, image transfer, video, and interactive multimedia. The emergence of new land-based and satellite-based wireless networks illustrates the expanding scale and trend toward globalization and the need to establish new local exchange and exchange access services to meet the communications needs of people on the move. An important issue is to identify the roles that satellite networking can play in meeting these new communications needs. The unique capabilities of satellites, in providing coverage to large geographic areas, reaching widely dispersed users, for position location determination, and in offering broadcast and multicast services, can complement and extend the capabilities of terrestrial networks. As an initial step in exploring the opportunities afforded by the merger of satellite-based and land-based networks, several experiments utilizing the NASA ACTS satellite and the public switched local exchange network were undertaken to demonstrate the use of satellites in the delivery of personal communications services.

Wolff, Richard S.↗
NASA NTRS2006

Biology Inspired Approach for Communal Behavior in Sensor Networks

Research in wireless sensor network technology has exploded in the last decade. Promises of complex and ubiquitous control of the physical environment by these networks open avenues for new kinds of science and business. Due to the small size and low cost of sensor devices, visionaries promise systems enabled by deployment of massive numbers of sensors working in concert. Although the reduction in size has been phenomenal it results in severe limitations on the computing, communicating, and power capabilities of these devices. Under these constraints, research efforts have concentrated on developing techniques for performing relatively simple tasks with minimal energy expense assuming some form of centralized control. Unfortunately, centralized control does not scale to massive size networks and execution of simple tasks in sparsely populated networks will not lead to the sophisticated applications predicted. These must be enabled by new techniques dependent on local and autonomous cooperation between sensors to effect global functions. As a step in that direction, in this work we detail a technique whereby a large population of sensors can attain a global goal using only local information and by making only local decisions without any form of centralized control.

Jones, Kennie H.↗