Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,351 records · Page 75
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Machine Learning Methods for Connection RTT and Loss Rate Estimation Using MPI Measurements Under Random Losses

Scientific computations are expected to be increasingly distributed across wide-area networks, and Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. Application-level measurements of MPI operations reflect the connection Round-Trip Time (RTT) and loss rate, and machine learning methods have been previously developed to estimate them under deterministic periodic losses. In this paper, we consider more complex, random losses with unform, Poisson and Gaussian distributions. We study five disparate machine leaning methods, with linear and non-linear, and smooth and non-smooth properties, to estimate RTT and loss rate over 10Gbps connections with 0-366ms RTT. The diversity and complexity of these estimators combined with the randomness of losses and TCP’s non-linear response together rule out the selection of a single best among them; instead, we fuse them to retain their design diversity. Overall, the results show that accurate estimates can be generated at low loss rates but become inaccurate at loss rates 10% and higher, thereby illustrating both their strengths and limitations.

Generalization Bounds↗
DOE OSTI2020

Towards the design of a wideband reflective long period grating distributed sensor

Abstract In this paper, we computationally investigate the effects of metal coating length and coating coverage on the reflected spectrum of a long period grating (LPG) over a broad bandwidth. Simulation results indicate that coating the tail end of the fiber between the LPG and the end facet of the fiber provides a reflected spectrum that mimics the LPG transmission spectrum shape over a 400 nm bandwidth. Based on single LPG simulation results, we present the design of a distributed LPG structure containing a multiple number ( n ) of LPGs in reflection mode for the first time. Simulation results for n = 1, 2, and 3 are presented here to demonstrate the concept of a distributed reflective LPG design. It is expected that such a sensor will open a new window for distributed sensing using reflective LPGs.

Rana, Sohel (ORCID:0000000262813555)↗
DOE OSTI2017

Optimizing checkpoint data placement with guaranteed burst buffer endurance in large-scale hierarchical storage systems

Non-volatile devices, such as SSDs, will be an integral part of the deepening storage hierarchy on large-scale HPC systems. These devices can be on the compute nodes as part of a distributed burst buffer service or they can be external. Wherever they are located in the hierarchy, one critical design issue is the SSD endurance under the write-heavy workloads, such as the checkpoint I/O for scientific applications. For these environments, it is widely assumed that checkpoint operations can occur once every 60 min and for each checkpoint step as much as half of the system memory can be written out. Unfortunately, for large-scale HPC applications, the burst buffer SSDs can be worn out much more quickly given the extensive amount of data written at every checkpoint step. One possible solution is to control the amount of data written by reducing the checkpoint frequency. However, a direct effect caused by reduced checkpoint frequency is the increased vulnerability window of system failures and therefore potentially wasted computation time, especially for large-scale compute jobs.

Computer Science↗
DOE OSTI2022

Impact of cyber attacks on distributed compressive sensing based state estimation in power distribution grids

Modern power distribution grids suffer from multiple vulnerabilities due to the tight integration between the physical system and the cyber infrastructure. Sophisticated and malicious cyber attacks continue to adversely impact the grid operation leading to performance degradation, service interruption, and grid failure. State estimation plays an essential role in grid monitoring and advancing cyber-attack situational awareness. In this regard, this paper first proposes a distributed compressive sensing (CS) state estimation approach for an unobservable distribution grid. Further, the proposed distributed CS approach divides the distribution grid into sub-areas to perform local state estimation. Then an alternating direction method of multipliers (ADMM) based iterative information exchange among neighboring areas is employed to complete the estimation process. In this estimation process, the impact of loss of measurement data, false data injection (FDI), replay, and neighborhood cyber-attacks is analyzed. Extensive simulations are performed on the IEEE 37-bus and IEEE 123-bus standard networks to demonstrate the algorithm’s robustness to the aforementioned cyber-attacks. A quantitative analysis of computational complexity and simulation time of the distributed CS based approach is also presented.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Planning Roadmap for DER Integration in India: Industry Best Practices and Resource Guide

Ensuring safe, reliable, cost-effective DER integration at scale requires holistic planning, broad stakeholder engagement, and should address key development areas such as standards adoption, equipment testing and certification, interoperability and cybersecurity, interconnection procedures, and advanced forecasting and DER management. Each of these areas currently represent significant challenges for utilities, regulators, OEMs, developers, and even consumers worldwide. India has already seen significant growth of DERs and has announced targets for substantial growth yet to come, with the potential for DERs to make up a non-negligible portion of the country's overall generation capacity. As such, it is of critical importance that Indian stakeholders consider the potential impacts of wide-spread adoption of DERs and take preemptive action related to the five development areas listed here, among others. India should consider strategies including the adoption of key DER standards related to interconnection, testing, and cybersecurity; enabling effective and secure communication channels for DER interoperability; testing and certifying DER equipment in accredited testing laboratories; building robust, streamlined interconnection procedures; and revamping legacy system planning structures to incorporate DERs in a holistic planning framework.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Powered By CADET

The Capacity Expansion Decision Support for Distribution Networks (CADET) is a Python-based library and framework for creating electrical distribution system capacity planning tools for cost-effective, reliable power delivery. It enables the creation of modular, scalable, and extensible distribution capacity planning tools by providing a high-level optimization interface, parameter and options data managers, optimization constraint and objective libraries, generalized nomenclature, a system for tracking and modifying distribution network changes, optimization solution validation, and other capabilities. This webinar will describe 1) the motivation for creating CADET, 2) key designs, and 3) several use cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2015

Direct Observation of Quark-Hadron Duality in the Free Neutron {ital F}{sub 2} Structure Function

Using the recently published data from the BONuS(Barely Off-shell Nucleon Structure) experiment at Jefferson Lab, which utilized a spectator tagging technique to extract the inclusive electron-free neutron scattering cross section, we obtain the first direct observation of quark-hadron duality in the neutron F-2 structure function. The data are used to reconstruct the lowest few (N = 2, 4, and 6) moments of F-2 in the three prominent nucleon resonance regions, as well as the moments integrated over the entire resonance region. Comparison with moments computed from global parametrizations of parton distribution functions suggest that quark-hadron duality holds locally for the neutron in the second and third resonance regions down to Q(2) approximate to 1 GeV2, with violations possibly up to 20% observed in the first resonance region.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Fast Tuning-Free Distributed Algorithm for Solving the Network-Constrained Economic Dispatch

With the increasing penetration of distributed energy resources (DERs) and their participation in the electricity market, it becomes more desirable to apply distributed algorithms for resource allocation in order to address the resulting computational and communicational challenges. Most of the existing distributed algorithms for solving the network-constrained economic dispatch (NCED) problem require the tuning of certain auxiliary parameters. As a result, the robustness of these algorithms against the varieties in DERs is greatly undermined. In this paper, a new distributed algorithm, optimality condition consensus (OCC), is proposed to solve the NCED problem by using distributed power flow (DPF) and ratio consensus as fundamental tools. It inherits the advantages of existing distributed algorithms for the NCED problem but removes the need for parameter tuning to improve performance in practice. In conclusion, the effectiveness of the proposed distributed algorithm in terms of efficiency, scalability, and robustness is demonstrated through detailed case studies.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2014

GADRAS-DRF User's Manual (V.18.5)

The Gamma Detector Response and Analysis Software - Detector Response Function (GADRAS-DRF) application computes the response of gamma-ray and neutron detectors to incoming radiation. This manual provides step-by-step procedures to acquaint new users with the use of the application. The capabilities include characterization of detector response parameters, plotting and viewing measured and computed spectra, analyzing spectra to identify isotopes, and estimating source energy distributions from measured spectra. GADRAS-DRF can compute and provide detector responses quickly and accurately, giving users the ability to obtain usable results in a timely manner (a matter of seconds or minutes).

Horne, Steven M.↗
DOE OSTI2016

GADRAS-DRF 18.6 User's Manual

The Gamma Detector Response and Analysis Software–Detector Response Function (GADRAS-DRF) application computes the response of gamma-ray and neutron detectors to incoming radiation. This manual provides step-by-step procedures to acquaint new users with the use of the application. The capabilities include characterization of detector response parameters, plotting and viewing measured and computed spectra, analyzing spectra to identify isotopes, and estimating source energy distributions from measured spectra. GADRAS-DRF can compute and provide detector responses quickly and accurately, giving users the ability to obtain usable results in a timely manner (a matter of seconds or minutes).

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2015

GADRAS-DRF 18.5 User's Manual

The Gamma Detector Response and Analysis Software--Detector Response Function (GADRAS-DRF) application computes the response of gamma-ray and neutron detectors to incoming radiation. This manual provides step-by-step procedures to acquaint new users with the use of the application. The capabilities include characterization of detector response parameters, plotting and viewing measured and computed spectra, analyzing spectra to identify isotopes, and estimating source energy distributions from measured spectra. GADRAS-DRF can compute and provide detector responses quickly and accurately, giving users the ability to obtain usable results in a timely manner (a matter of seconds or minutes).

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2016

THE EFFECTS OF GRAIN SIZE AND TEMPERATURE DISTRIBUTIONS ON THE FORMATION OF INTERSTELLAR ICE MANTLES

Computational models of interstellar gas-grain chemistry have historically adopted a single dust-grain size of 0.1 micron, assumed to be representative of the size distribution present in the interstellar medium. Here, we investigate the effects of a broad grain-size distribution on the chemistry of dust-grain surfaces and the subsequent build-up of molecular ices on the grains, using a three-phase gas-grain chemical model of a quiescent dark cloud. We include an explicit treatment of the grain temperatures, governed both by the visual extinction of the cloud and the size of each individual grain-size population. We find that the temperature difference plays a significant role in determining the total bulk ice composition across the grain-size distribution, while the effects of geometrical differences between size populations appear marginal. We also consider collapse from a diffuse to a dark cloud, allowing dust temperatures to fall. Under the initial diffuse conditions, small grains are too warm to promote grain-mantle build-up, with most ices forming on the mid-sized grains. As collapse proceeds, the more abundant, smallest grains cool and become the dominant ice carriers; the large population of small grains means that this ice is distributed across many grains, with perhaps no more than 40 monolayers of ice each (versus several hundred assuming a single grain size). This effect may be important for the subsequent processing and desorption of the ice during the hot-core phase of star formation, exposing a significant proportion of the ice to the gas phase, increasing the importance of ice-surface chemistry and surface–gas interactions.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Quantum Distributed Unit Commitment: An Application in Microgrids

The dawn of quantum computing brings on a revolution in the way combinatorially complex power system problems such as Unit Commitment are solved. The Unit Commitment problem complexity is expected to increase in the future because of the trend toward the increase of penetration of intermittent renewables. Even though quantum computing has proven effective for solving a host of problems, its applications for power systems’ problems have been rather limited. Here, in this paper, a quantum unit commitment is innovatively formulated and the quantum version of the decomposition and coordination alternate direction method of multipliers (ADMM) is established. The above is achieved by devising quantum algorithms and by exploiting the superposition and entanglement of quantum bits (qubits) for solving subproblems, which are then coordinated through ADMM to obtain feasible solutions. The main contributions of this paper include: 1) the innovative development of a quantum model for Unit Commitment; 2) development of decomposition and coordination-supported framework which paves the way for the utilization of limited quantum resources to potentially solve the large-scale discrete optimization problems; 3) devising the novel quantum distributed unit commitment (QDUC) to solve the problem in a larger scale than currently available quantum computers are capable of solving. The QDUC results are compared with those from its classical counterpart, which validate the efficacy of quantum computing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Diaspora: Resilience-Enabling Services for Real-Time Distributed Workflows

The need for real-time processing to enable automated decision making and experimental steering has driven a shift from high-performance computing workflows on a centralized system to a distributed approach that integrates remote data sources, edge devices, and diverse compute facilities. Under this paradigm, data can be processed close to the source where it is generated, thus reducing latency and bandwidth usage. System resilience is thus a key challenge, requiring distributed workflows to survive component failures and to meet stringent quality-of-service requirements, which results in the need to mitigate anomalies such as congestion and low availability of resources. To address these challenges, we propose Diaspora, a unified resilience framework that is inspired by event-driven communication patterns used in public clouds. Specifically, we propose an event fabric that extends across sites, facilities, and computations to provide timely, reliable, and accurate information about data, application, and resource status. On top of the event fabric, we build resilience-enabling services that combine QoS-aware data streaming, resilient data views, resilient compute and data resources, and anomaly detection and prediction, all of which collectively enhance workflow resilience for these scientific cases.

Rao, Nageswara↗
DOE OSTI2024

Generalized parton distributions from lattice QCD with asymmetric momentum transfer: Axial-vector case

Recently, we made significant advancements in improving the computational efficiency of lattice QCD calculations for generalized parton distributions (GPDs). This progress was achieved by adopting calculations of matrix elements in asymmetric frames, deviating from the computationally-expensive symmetric frame typically used, and allowing freedom in the choice for the distribution of the momentum transfer between the initial and final states. A crucial aspect of this approach involves the adoption of a Lorentz covariant parametrization for the matrix elements, introducing Lorentz-invariant amplitudes. This approach also allows us to propose an alternative definition of quasi-GPDs, ensuring frame independence and potentially reduce power corrections in matching to light cone GPDs. In our previous work, we presented lattice QCD results for twist-2 unpolarized GPDs ( H and E ) of quarks obtained from calculations performed in asymmetric frames at zero skewness. Building upon this work, we now introduce a novel Lorentz covariant parametrization for the axial-vector matrix elements. We employ this parametrization to compute the axial-vector GPD H ˜ at zero skewness, using an N f = 2 + 1 + 1 ensemble of twisted mass fermions with clover improvement. The light-quark masses employed in our calculations correspond to a pion mass of approximately 260 MeV. Published by the American Physical Society 2024

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2018

Distributed Louvain Algorithm for Graph Community Detection

In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a community are more likely to be “related” to one another than they are to the rest of the network. The goodness of partitioning into communities is typically measured using a well known measure called modularity. However, modularity optimization is an NP-complete problem. In 2008, Blondel, et al. introduced a multi-phase, iterative heuristic for modularity optimization, called the Louvain method. Owing to its speed and ability to yield high quality communities, the Louvain method continues to be one of the most widely used tools for serial community detection. In this paper, we present the design of a distributed memory implementation of the Louvain algorithm for parallel community detection. Our approach begins with an arbitrarily partitioned distributed graph input, and employs several heuristics to speedup the computation of the different steps of the Louvain algorithm. We evaluate our implementation and its different variants using real-world networks from various application domains (including internet, biology, social networks). Our MPI+OpenMP implementation yields about 7x speedup (on 4K processes) for soc-friendster network (1.8B edges) over a state-of-the-art shared-memory multicore implementation (on 64 threads), without compromising output quality. Furthermore, our distributed implementation was able to process a larger graph (uk-2007; 3.3B edges) in 32 seconds on 1K cores (64 nodes) of NERSC Cori, when the state-of-the-art shared-memory implementation failed to run due to insufficient memory on a single Cori node containing 128 GB.

High performance computing, graph algorithms, Dist↗
DOE OSTI2021

Federated Quantum Machine Learning

Distributed training across several quantum computers could significantly improve the training time and if we could share the learned model, not the data, it could potentially improve the data privacy as the training would happen where the data is located. One of the potential schemes to achieve this property is the federated learning (FL), which consists of several clients or local nodes learning on their own data and a central node to aggregate the models collected from those local nodes. However, to the best of our knowledge, no work has been done in quantum machine learning (QML) in federation setting yet. In this work, we present the federated training on hybrid quantum-classical machine learning models although our framework could be generalized to pure quantum machine learning model. Specifically, we consider the quantum neural network (QNN) coupled with classical pre-trained convolutional model. Our distributed federated learning scheme demonstrated almost the same level of trained model accuracies and yet significantly faster distributed training. It demonstrates a promising future research direction for scaling and privacy aspects.

97 MATHEMATICS AND COMPUTING↗