Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Phase Demodulation by Frequency Chirping in Coherence Microwave Photonic Interferometry

This paper presents a signal processing method to demodulate the optical interference phase of cascaded individual optical fiber intrinsic Fabry-Perot interferometric (IFPI) sensors in a coherent microwave-photonic interferometry (CMPI) distributed sensing system. The new method utilizes the chirp effect of electro-optic modulator (EOM) to create a quasi-quadrature optical interference phase shift between two adjacent pulses which correspond to two adjacent reflection points in the time domain. The phase shift can be controlled by adjusting the bias voltage that is applied to the EOM. The interference phase is calculated by elliptically fitting the phase shift. The interference phase change is proportional to the optical path difference (OPD) change of the interferometer, and the sign can be used to differentiate the increase or decrease of the OPD. The method is demonstrated for distributed strain sensing, showing good linearity, high resolution and large dynamic range.

42 ENGINEERING↗
DOE OSTI2020

Exploiting CRIC to streamline the configuration management of GlideinWMS factories for CMS support

GlideinWMS is a workload management and provisioning system that allows sharing computing resources distributed over independent sites. Based on the requests made by GlideinWMS frontends, a dynamically sized pool of resources is created by GlideinWMS pilot factories via pilot job submission to resource sites’ CEs. More than 400 CEs are currently serving more than ten virtual organizations through GlideinWMS, with CMS being the biggest user with 230 CEs. The complex configurations of the parameters defining resource requests, as submitted to those CEs, have been historically managed by manually editing a set of different XML files. New possibilities arise with CMS adopting the CRIC, an information system that collects, aggregates, stores, and exposes, among other things, computing resource data coming from various data providers. The paper will describe the challenges faced when CMS started to use CRIC to automatically generate the GlideinWMS factory configurations. The architecture of the prototype, and the ancillary tools developed to ease this transition, will be discussed. Finally, future plans and milestones will be outlined.

Dost, Jeffrey↗
DOE OSTI2026

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Iteration-based Linearized Distribution-level Locational Marginal Price for Three-phase Unbalanced Distribution Systems

Distributed energy resources (DERs) are rocking the utilities’ business landscape. It calls for competitive market environments that incentivize DERs to form maximum operating efficiency. Among proposed pricing schemes, distribution-level locational marginal price (DLMP) is effective in signaling the marginal generation cost differences driven by energy losses and network constraints. It can be derived from a distribution-level optimal power flow (OPF) framework, as it essentially presents the sensitivity of optimized generation cost towards incremental loads. However, due to the high resistance-to-inductance ratio and unbalanced characteristics of distribution networks, computational affordable DLMPs are highly challenged. This article provides a linear-approximated DLMP that can be solved efficiently and generalized to account for reactive power flow, three-phase unbalanced loads and meshed network structure. The successive linear programming technique is introduced to enhance the model accuracy. Case studies on an IEEE 123-Bus system validate its accuracy against a nonlinear benchmark and capability in offering proper incentives.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Generating Massive Scale-free Networks: Novel Parallel Algorithms using the Preferential Attachment Model

Recently, there has been substantial interest in the study of various random networks as mathematical models of complex systems. As real-life complex systems grow larger, the ability to generate progressively large random networks becomes all the more important. This motivates the need for efficient parallel algorithms for generating such networks. Naïve parallelization of sequential algorithms for generating random networks is inefficient due to inherent dependencies among the edges and the possibility of creating duplicate (parallel) edges. In this article, we present message passing interface-based distributed memory parallel algorithms for generating random scale-free networks using the preferential-attachment model. Our algorithms are experimentally verified to scale very well to a large number of processing elements (PEs), providing near-linear speedups. The algorithms have been exercised with regard to scale and speed to generate scale-free networks with one trillion edges in 6 minutes using 1,000 PEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimizing High-Throughput Inference on Graph Neural Networks at Shared Computing Facilities with the NVIDIA Triton Inference Server

Abstract With machine learning applications now spanning a variety of computational tasks, multi-user shared computing facilities are devoting a rapidly increasing proportion of their resources to such algorithms. Graph neural networks (GNNs), for example, have provided astounding improvements in extracting complex signatures from data and are now widely used in a variety of applications, such as particle jet classification in high energy physics (HEP). However, GNNs also come with an enormous computational penalty that requires the use of GPUs to maintain reasonable throughput. At shared computing facilities, such as those used by physicists at Fermi National Accelerator Laboratory (Fermilab), methodical resource allocation and high throughput at the many-user scale are key to ensuring that resources are being used as efficiently as possible. These facilities, however, primarily provide CPU-only nodes, which proves detrimental to time-to-insight and computational throughput for workflows that include machine learning inference. In this work, we describe how a shared computing facility can use the NVIDIA Triton Inference Server to optimize its resource allocation and computing structure, recovering high throughput while scaling out to multiple users by massively parallelizing their machine learning inference. To demonstrate the effectiveness of this system in a realistic multi-user environment, we use the Fermilab Elastic Analysis Facility augmented with the Triton Inference Server to provide scalable and high-throughput access to a HEP-specific GNN and report on the outcome.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

T-FSM: A Scalable Distributed Task-Based System for Frequent Subgraph Pattern Mining from a Big Graph

Finding frequent subgraph patterns in a big graph is an important problem with many applications such as classifying chemical compounds and building indexes to speed up graph queries. Since this problem is NP-hard, some recent parallel and distributed systems have been developed to accelerate the mining. However, they often have a huge memory cost, very long running time, suboptimal load balancing, poor scale-out capability, and possibly inaccurate results. In this article, we propose an efficient system called T-FSM for parallel mining of frequent subgraph patterns in a big graph. T-FSM supports a new anti-monotonic frequentness measure called Fraction-Score, which is more accurate than the widely used MNI measure. The execution engine of T-FSM supports both intra-machine parallelism and inter-machine parallelism. For intra-machine parallelism, T-FSM adopts a novel task-based execution model to ensure high multithreading concurrency, bounded memory consumption, and effective load balancing. For inter-machine parallelism, T-FSM ensures good scale-out performance with a lightweight pattern rebalancing approach that reduces workload skewness of pattern evaluations among machines. To avoid recomputing the contexts for migrated patterns, we design a novel context cache table to support concurrent and asynchronous requesting and caching of remote context data, which can timely evict and garbage collect used pattern contexts that are no longer needed to keep memory consumption bounded. Extensive experiments show that T-FSM is orders of magnitude faster than existing state-of-the-art parallel systems (more than 10×, 51×, 131×, 55× speedup over ScaleMine, DistGraph, Pangolin and Peregrine, respectively) and distributed systems (more than 42× and 88× over ScaleMine and DistGraph, respectively) for frequent subgraph pattern mining, and it scales out satisfactorily to 512 CPU cores on the Polaris supercomputer at Argonne National Laboratory.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2010

Polyphony: A Workflow Orchestration Framework for Cloud Computing

Cloud Computing has delivered unprecedented compute capacity to NASA missions at affordable rates. Missions like the Mars Exploration Rovers (MER) and Mars Science Lab (MSL) are enjoying the elasticity that enables them to leverage hundreds, if not thousands, or machines for short durations without making any hardware procurements. In this paper, we describe Polyphony, a resilient, scalable, and modular framework that efficiently leverages a large set of computing resources to perform parallel computations. Polyphony can employ resources on the cloud, excess capacity on local machines, as well as spare resources on the supercomputing center, and it enables these resources to work in concert to accomplish a common goal. Polyphony is resilient to node failures, even if they occur in the middle of a transaction. We will conclude with an evaluation of a production-ready application built on top of Polyphony to perform image-processing operations of images from around the solar system, including Mars, Saturn, and Titan.

Space Exploration,↗
NASA NTRSDate not supplied

Optimal Communication Topology Determination and Sensor Selection for Independent Airspace Surveillance

The paper presents an approach to sensors selection and network topology determination for independent airspace surveillance with maximum outcome and minimum cost using ground based distributed sensing, computing and communication network infrastructure. The selection criteria includes minimum estimation error, maximum airspace coverage, minimum communication time and power consumption while guaranteeing the system observability and providing in-time quality information to a monitoring observer. The developed algorithm uses multi-objective optimization strategy taking into account trade-offs between conflicting objectives and relaxations for in time implementation. It is implemented utilizing graph theoretic tools. The approach is validated in a desktop simulation environment using synthetic sensors data generated for a simulated multi-vehicle flight scenario in the selected regional airspace.

Distributed sensing↗
NASA NTRSDate not supplied

Separation and Transition on the ROTEX-T Cone-Flare

As part of NATO STO AVT-346 “Predicting Hypersonic Boundary-Layer Transition on Complex Geometries,” coordinated experimental and computational studies were conducted on the ROTEX-T, a cone-flare geometry used in a successful flight-test experiment. At the as-flown conditions, a separation bubble existed at the compression corner. Separation, reattachment, and the multifaceted linear instability paths leading this bubble to transition to turbulence are challenging to predict, but have significant impact on surface pressure and heat flux. High-resolution background-oriented schlieren and infrared thermography measurements were made in the AFOSR–Notre Dame Large Mach-6 Quiet Tunnel at freestream unit Reynolds numbers from 5.8×10(exp 6) to 12.2×10(exp 6) m-1and nominally zero angle of attack. High-speed self-aligned focusing schlieren, infrared thermography, and focused laser differential interferometry measurements were made in the AFRL Mach-6 Ludwieg Tube from 2.2×10(exp 6) to 24.7×10(exp 6) m-1and nominally zero degrees angle of attack. The surface heat-flux and Stanton number distributions were computed. Separation and reattachment locations, as well as the flow state at each, were determined from the combination of surface and off-wall measurements. The convective and global boundary-layer instabilities of the axisymmetric laminar flow at the experimental conditions were investigated computationally. Amplification of Mack’s first and second modes were observed to have logarithmic amplification factors between 5 to 7.5 at the separation location, depending on conditions. The flow was found to be globally unstable to stationary three-dimensional disturbances concentrated in the reattachment region. Previous analysis of the ROTEX-T flight data had not assessed reattachment location or the flow state upon reattachment. Thanks to the insights gained from the coordinated, on- and off-wall ground-test measurements, these evaluations have now been made. The separation location indicated by laminar simulations is consistently numerically predicted to be upstream of the experimentally observed location for a transitional separation bubble. The cause of this difference is understood to lie in the steady-state and axisymmetric assumptions made by both solvers employed to compute the basic states analyzed, as flow topology considerations assert that unsteady two-dimensional or axisymmetric separation bubbles are structurally unstable and will become three-dimensional. Computed laminar heating rates prior to separation agreed well with experiment; transitional heating rates after reattachment were between laminar and turbulent computations.

CFD Validation↗
NASA NTRSDate not supplied

Separation and Transition on the ROTEX-T Cone-Flare

As part of NATO STO AVT-346 “Predicting Hypersonic Boundary-Layer Transition on Complex Geometries,” coordinated experimental and computational studies were conducted on the ROTEX-T, a cone-flare geometry used in a successful flight-test experiment. At the as-flown conditions, a separation bubble existed at the compression corner. Separation, reattachment, and the multifaceted linear instability paths leading this bubble to transition to turbulence are challenging to predict, but have significant impact on surface pressure and heat flux. High-resolution background-oriented schlieren and infrared thermography measurements were made in the AFOSR–Notre Dame Large Mach-6 Quiet Tunnel at freestream unit Reynolds numbers from 5.8×10(exp 6) to 12.2×10(exp 6) m-1and nominally zero angle of attack. High-speed self-aligned focusing schlieren, infrared thermography, and focused laser differential interferometry measurements were made in the AFRL Mach-6 Ludwieg Tube from 2.2×10(exp 6) to 24.7×10(exp 6) m-1and nominally zero degrees angle of attack. The surface heat-flux and Stanton number distributions were computed. Separation and reattachment locations, as well as the flow state at each, were determined from the combination of surface and off-wall measurements. The convective and global boundary-layer instabilities of the axisymmetric laminar flow at the experimental conditions were investigated computationally. Amplification of Mack’s first and second modes were observed to have logarithmic amplification factors between 5 to 7.5 at the separation location, depending on conditions. The flow was found to be globally unstable to stationary three-dimensional disturbances concentrated in the reattachment region. Previous analysis of the ROTEX-T flight data had not assessed reattachment location or the flow state upon reattachment. Thanks to the insights gained from the coordinated, on- and off-wall ground-test measurements, these evaluations have now been made. The separation location indicated by laminar simulations is consistently numerically predicted to be upstream of the experimentally observed location for a transitional separation bubble. The cause of this difference is understood to lie in the steady-state and axisymmetric assumptions made by both solvers employed to compute the basic states analyzed, as flow topology considerations assert that unsteady two-dimensional or axisymmetric separation bubbles are structurally unstable and will become three-dimensional. Computed laminar heating rates prior to separation agreed well with experiment; transitional heating rates after reattachment were between laminar and turbulent computations.

CFD Validation↗
DOE OSTI2025

Clustering at Massive Scale

ClaMS provides hierarchical clustering technology for use on massive, high-dimensional datasets that require distributed memory for processing. The algorithm employed is inspired by the popular HDBSCAN algorithm but makes use of computational kernels better suited for distributed computing. ClaMS is built on scalable nearest neighbor graph construction, metric forest completion, and approximate minimum spanning tree techniques.

Stanley, ThomasA [Lawrence Livermore National Labo↗
DOE OSTI2023

Modular chip-integrated photonic control of artificial atoms in diamond waveguides

A central goal in creating long-distance quantum networks and distributed quantum computing is the development of interconnected and individually controlled qubit nodes. Atom-like emitters in diamond have emerged as a leading system for optically networked quantum memories, motivating the development of visible-spectrum, multi-channel photonic integrated circuit (PIC) systems for scalable atom control. However, it has remained an open challenge to realize optical programmability with a qubit layer that can achieve high optical detection probability over many optical channels. Here, we address this problem by introducing a modular architecture of piezoelectrically actuated atom-control PICs (APICs) and artificial atoms embedded in diamond nanostructures designed for high-efficiency free-space collection. The high-speed four-channel APIC is based on a splitting tree mesh with triple-phase shifter Mach–Zehnder interferometers. This design simultaneously achieves optically broadband operation at visible wavelengths, high-fidelity switching (>40dB) at low voltages, submicrosecond modulation timescales (>30MHz), and minimal channel-to-channel crosstalk for repeatable optical pulse carving. Via a reconfigurable free-space interconnect, we use the APIC to address single silicon vacancy color centers in individual diamond waveguides with inverse tapered couplers, achieving efficient single photon detection probabilities (∼15%) and second-order autocorrelation measurements g (2) (0)<0.14 for all channels. The modularity of this distributed APIC–quantum memory system simplifies the quantum control problem, potentially enabling further scaling to thousands of channels.

47 OTHER INSTRUMENTATION↗
NASA NTRS2014

Cultivating an Emergent Earth Observation Analytics Ecosystem in the Cloud

A diverse set of data analytics systems for Earth Observations are sprouting up in the Earth Science community, with a wealth of processing algorithms and analysis methods. There is a similar wealth of data resources available via myriad data providers and clearinghouses, including large institutional systems like the Earth Observing System Data and Information System, Comprehensive Large Scale Array-data Stewardship System, and Federated Earth Observation Missions gateway. With Earth system science driving a need to work with more datasets together, and the community developing more analysis tools (some of them dataset-specific), how can we develop analysis workflows that incorporate far-flung datasets and leverage analysis resources from multiple organizations? Cloud computing points the way toward a solution in two different respects. Firstly, the access to and abstraction of virtually unlimited storage and computing power provides an environment that enables more straightforward means of pulling datasets and analysis resources together. Just as importantly, however, cloud computing serves as an example of an "ecosystem" of interoperating services, since the essence of cloud computing is the presentation of all resources as a service, from hardware to infrastructure to platform to software. This enables the combination of off-the-shelf, diverse services to construct entire systems that emerge out of an equally diverse community of architects and developers. This approach can be similarly applied to the data and analysis resources in the Earth Observation community. By exposing these resources via well understood services, and consuming resources in the same way, different organizations can construct bespoke analysis workflows and systems for their own purposes. The key leap the community needs to make is to develop analysis systems in components that interact with other components via services. The result would be a rich ecosystem of analytics components that can be combined to analyze datasets at scale and in conjunction with other datasets from other sources.

chaos↗
DOE OSTI2021

Non-Diffusive Volume Advection with A High Order Interface Reconstruction Method

We show that non-diffusive volume advection in two-dimensions is achieved with several benchmark problems using a newly developed high-order volume of fluids (VOF) interface reconstruction method. (1) A new VOF interface reconstruction method using circular/corner facets (linear facets are a degenerate case of arcs). We create a circular interface facet in each mixed zone by matching neighbor volume with a hybrid Newton’s-bisection method and the local solution is final. In the general case, the new VOF interface reconstruction has 3rd order accuracy and can be easily made seamless. The new method addresses intrinsic issues with Young’s method such as gaps between interface facets in the case of a curved interface, and inability to define curvature nor identify corners. (2) A non-diffusive volume advection scheme. In an ALE advection step, a well-defined interface can be carried over through a Lagrange step and used to compute volume distribution into a relaxed mesh. Then, an interface reconstruction step is performed to redefine the interface in the relaxed mesh. We must point out that the interface carried over is also a solution of interface reconstruction because all the volume fractions in the relaxed mesh are naturally matched. We provide an interface tracking method compatible with our reconstruction scheme, where it is granted to use the prior info as an initial guess to capture sub-mesh resolution features. As a result, we are able to treat multiple facets inside a single mixed cell and obtain highly accurate, non-diffusive solution for advection problems with rather coarse meshes. We show our solutions for two-dimensional incompressible flows with two materials with a) the X + O diagonal translation; b) the Zalesak rotational test; and c) the single vortex spiral test.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

Fast Multilevel Implementation of Recursive Spectral Bisection for Partitioning Unstructured Problems

If problems involving unstructured meshes are to be solved efficiently on distributed-memory parallel computers, the meshes must be partitioned and distributed across processors in a way that balances tile computational load and minimizes communication. The recursive spectral bisection method (RSB) has been shown to be very effective for such partitioning problems compared to alternative methods, but RSB in its simplest form is expensive. Here a multilevel version of RSB is introduced that attains about an order-of-magnitude improvement in run time on typical examples.

Barnard, Stephen T.↗
DOE OSTI2020

Assessing DER Network Cybersecurity Defences in a Power-Communication Co-Simulation Environment

Increasing penetrations of interoperable distributed energy resources (DER) in the electric power system are expanding the power system attack surface. Maloperation or malicious control of DER equipment can now cause substantial disturbances to grid operations. Fortunately, many options exist to defend and limit adversary impact on these newly-created DER communication networks, which typically traverse the public internet. However, implementing these security features will increase communication latency, thereby adversely impacting real-time DER grid support service effectiveness. In this work, a collection of software tools called SCEPTRE were used to create a co-simulation environment where SunSpec-compliant PV inverters were deployed as virtual machines and interconnected to simulated communication network equipment. Network segmentation, encryption, and moving target defence security features were deployed on the control network to evaluate their influence on cybersecurity metrics and power system performance. The results indicated that adding these security features did not impact DER-based grid control systems but improved the cybersecurity posture of the network when implemented appropriately.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Forecasting Dynamic Line Rating with Spatial Variation Considerations

Dynamic line rating (DLR) is a technology that allows the ampacity of an electrical conductor to be calculated using real-time or forecasted weather conditions. Historically, the ampacity of a conductor has been determined using a static line rating method which assumes conservative weather assumptions. Therefore, not only can DLR give a more accurate measurement of the true ampacity of a conductor, but it can also increase its ampacity during weather conditions with greater thermal mitigations. The two primary cooling factors in the ampacity calculations are wind speed and direction. In complex terrain, wind speed and direction can have large variations over short distances. Therefore, accurately identifying the limiting span of a transmission line requires high spatial resolution of the wind along its path. One solution is to install dense weather stations along their path, though this can become costly over long distances. Therefore, researchers have investigated the use of Computation Fluid Dynamic (CFD) simulations to accurately compute the wind field along the path of a transmission line and use these results to identify the limiting section of the conductor. This work presents a case study that evaluates the coupling of CFD simulations and forecasted weather simulations using the High-Resolution Rapid Refresh (HRRR) model points over a 2-year span within a region in south eastern Idaho. The primary goal of the work is the evaluation of the number of HRRR model points used, i.e., weather stations, along the path of the line and the accuracy of the resulting ampacity. This was done using 4, 10, 17, 26, and 35 HRRR model points along two transmission line paths. The results indicate that as the number of model points are increased, the DLR ampacity of the lines decrease, yet converge as more points are added and demonstrate little change with additional HRRR points. It is expected that these results can help transmission line operators identify the number of weather stations that must be installed when coupled with CFD simulations and DLR ampacity to ensure accurate ratings and safe operations.

17 WIND ENERGY↗