Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Sample IEEE123 Bus system for OEDI SI

Time series load and PV data from an IEEE123 bus system. An example electrical system, named the OEDI SI feeder, is used to test the workflow in a co-simulation. The system used is the IEEE123 test system, which is a well studied test system (see link below to IEEE PES Test Feeder), but some modifications were made to it to add some solar power modules and measurements on the system. The aim of this project is to create an easy-to-use platform where various types of analytics can be performed on a wide range of electrical grid datasets. The aim is to establish an open-source library of algorithms that universities, national labs and other developers can contribute to which can be used on both open-source and proprietary grid data to improve the analysis of electrical distribution systems for the grid modeling community. OEDI Systems Integration (SI) is a grid algorithms and data analytics API created to standardize how data is sent between different modules that are run as part of a co-simulation. The readme file included in the S3 bucket provides information about the directory structure and how to use the algorithms. The sensors.json file is used to define the measurement locations.

123 bus↗
DOE OSTI2023

Approximate Boltzmann distributions in quantum approximate optimization

Approaches to compute or estimate the output probability distributions from the quantum approximate optimization algorithm (QAOA) are needed to assess the likelihood it will obtain a quantum computational advantage. We analyze output from QAOA circuits solving 7200 random MaxCut instances, with $n$ = 14–23 qubits and depth parameter $p$ ≤ 12 and find that the average basis state probabilities follow approximate Boltzmann distributions: The average probabilities scale exponentially with their energy (cut value), with a peak at the optimal solution. Furthermore, we describe the rate of exponential scaling or effective temperature in terms of a series with a leading-order term $T$ ~ $C$ min /$n$ $\sqrt{p}$, with $C$ min the optimal solution energy. Using this scaling, we generate approximate output distributions with up to 38 qubits and find these give accurate accounts of important performance metrics in cases we can simulate exactly.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Adversarial sampling of unknown and high-dimensional conditional distributions

Many engineering problems require the prediction of realization-to-realization variability or a refined description of modeled quantities. In that case, it is necessary to sample elements from unknown high-dimensional spaces with possibly millions of degrees of freedom. While there exist methods able to sample elements from probability density functions (PDF) with known shapes, several approximations need to be made when the distribution is unknown. In this paper the sampling method, as well as the inference of the underlying distribution, are both handled with a data-driven method known as generative adversarial networks (GAN), which trains two competing neural networks to produce a network that can effectively generate samples from the training set distribution. In practice, it is often necessary to draw samples from conditional distributions. When the conditional variables are continuous, only one (if any) data point corresponding to a particular value of a conditioning variable may be available, which is not sufficient to estimate the conditional distribution. This work handles this problem using an a priori estimation of the conditional moments of a PDF. Herein, two approaches, stochastic estimation, and an external neural network are compared for computing these moments; however, any preferred method can be used. The algorithm is demonstrated in the case of the deconvolution of a filtered turbulent flow field. It is shown that all the versions of the proposed algorithm effectively sample the target conditional distribution with minimal impact on the quality of the samples compared to state-of-the-art methods. Additionally, the procedure can be used as a metric for the diversity of samples generated by a conditional GAN (cGAN) conditioned with continuous variables.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Butterfly Factorization Via Randomized Matrix-Vector Multiplications

This paper presents an adaptive randomized algorithm for computing the butterfly factorization of an m × n matrix with m ≈ n provided that both the matrix and its transpose can be rapidly applied to arbitrary vectors. The resulting factorization is composed of O(log n) sparse factors, each containing O(n) nonzero entries. The factorization can be attained using O(n 3/2 log n) computation and O(n log n) memory resources. Furthermore, the proposed algorithm can be implemented in parallel and can apply to matrices with strong or weak admissibility conditions arising from surface integral equation solvers as well as multi-frontal-based finite-difference, finite-element, or finite-volume solvers. A distributed-memory parallel implementation of the algorithm demonstrates excellent scaling behavior.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Recovering non-Maxwellian particle velocity distribution functions from collective Thomson-scattered spectra

Collective optical Thomson scattering (TS) is a diagnostic commonly used to characterize plasma parameters. These parameters are typically extracted by a fitting algorithm that minimizes the difference between a measured scattered spectrum and an analytic spectrum calculated from the velocity distribution function (VDF) of the plasma. However, most existing TS analysis algorithms assume that the VDFs are Maxwellian, and applying an algorithm that makes this assumption does not accurately extract the plasma parameters of a non-Maxwellian plasma due to the effect of non-Maxwellian deviations on the TS spectra. We present new open-source numerical tools for forward modeling analytic spectra from arbitrary VDFs and show that these tools are able to more accurately extract plasma parameters from synthetic TS spectra generated by non-Maxwellian VDFs compared to standard TS algorithms. Estimated posterior probability distributions of fits to synthetic spectra for a variety of example non-Maxwellian VDFs are used to determine uncertainties in the extracted plasma parameters and show that correlations between parameters can significantly affect the accuracy of fits in plasmas with non-Maxwellian VDFs.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

GPU-based Image Compression for Efficient Compositing in Distributed Rendering Applications

Visualizations of large-scale data sets are often created on graphics clusters that distribute the rendering task amongst many processes. When using real-time GPU-based graphics algorithms, the most time-consuming aspect of distributed rendering is typically the com-positing phase - combining all partial images from each rendering process into the final visualization. Compo siting requires image data to be copied off the GPU and sent over a network to other processes. While compression has been utilized in existing distributed rendering compositors to reduce the data being sent over the network, this compression tends to occur after the raw images are transferred from the GPU to main memory. In this paper, we present work that leverages OpenGL / CUDA interoperability to compress raw images on the GPU prior to transferring the data to main memory. This approach can significantly reduce the device-to-host data transfer time, thus enabling more efficient compositing of images generated by distributed rendering applications.

Lipinksi, Riley↗
DOE OSTI2022

Mitigating the Impacts of Uncertain Geomagnetic Disturbances on Electric Grids: A Distributionally Robust Optimization Approach

Severe geomagnetic disturbances (GMDs) increase the magnitude of the electric field on the Earth’s surface (E-field) and drive geomagnetically-induced currents (GICs) along the transmission lines in electric grids. These additional currents can pose severe risks, such as current distortions, transformer saturation and increased reactive power losses, each of which can lead to system unreliability. Today several mitigation actions (e.g., changing grid topology) exist that can reduce the harmful GIC effects on the grids. Making such decisions can be challenging, however, because the magnitude and direction of the E-field are uncertain and non-stationary. In this paper, we model uncertain E-fields using the distributionally robust optimization (DRO) approach that determines optimal transmission grid operations such that the worst-case expectation of the system cost is minimized. We also capture the effect of GICs on the nonlinear AC power flow equations. For solution approaches, we develop an accelerated column-and-constraint generation (CCG) algorithm by exploiting a special structure of the support set of uncertain parameters representing the E-field. Extensive numerical experiments based on “epri-21” and “uiuc-150” systems, designed for GMD studies, demonstrate (i) the computational performance of the accelerated CCG algorithm, (ii) the superior performance of distributionally robust grid operations that satisfy nonlinear, nonconvex AC power flow equations and GIC constraints, in comparison with standard stochastic programming-based methods during the out-of-sample testing.

42 ENGINEERING↗
DOE OSTI2020

Estimation of distributions via multilevel Monte Carlo with stratified sampling

We design and implement a novel algorithm for computing a multilevel Monte Carlo (MLMC) estimator of the joint cumulative distribution function (CDF) of a vector-valued quantity of interest in problems with random input parameters and initial conditions. Our approach combines MLMC with stratified sampling of the input sample space by replacing standard Monte Carlo at each level with stratified Monte Carlo initialized with proportionally allocated samples. We show that the resulting stratified MLMC (sMLMC) algorithm is more efficient than its standard MLMC counterpart due to the additional variance reduction provided by the stratification of the random parameter's domain, especially at the coarsest levels. Additional computational cost savings are obtained by smoothing the indicator function with a Gaussian kernel, which proves to be an efficient and robust alternative to recently developed polynomial-based techniques.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Distributed Coordination of Demand-side Flexible Resources in Microgrid with All-Time Feasibility

The prevalence of distributed renewable generators motivates microgrid operators to exploit demand-side flexible resources (DFRs). Due to their dispersed nature, distributed DFR coordination has been a vibrant research area, while there are several issues awaiting to be addressed. On one hand, DFR power is internally coupled through power flow, while DFR usually cannot access grid information. On the other hand, in time-restricted scenarios, solution feasibility cannot be guaranteed by conventional dual-based algorithms. To fill these gaps, we propose a distributed DFR coordination framework with all-time feasibility. The proposed framework accounts for the distinct access of microgrid entities to grid information. A distributed and all-time feasible algorithm is proposed for optimal DFR coordination, which allows DFRs to make local decisions without violating constraints throughout iterations. The effectiveness of the proposed algorithm is demonstrated through case studies. The impact of peer-to-peer communication links on algorithm convergence is also investigated, which emphasizes the balance between communication investment and algorithm performance.

Li, Hongyi [Iowa State Univ., Ames, IA (United Sta↗
DOE OSTI2023

Distributed Energy Management for Networked Microgrids with Hardware-in-the-Loop Validation

For the cooperative operation of networked microgrids, a distributed energy management considering network operational objectives and constraints is proposed in this work. Considering various ownership and privacy requirements of microgrids, utility directly interfaced distributed energy resources (DERs) and demand response, a distributed optimization is proposed for obtaining optimal network operational objectives with constraints satisfied through iteratively updated price signals. The alternating direction method of multipliers (ADMM) algorithm is utilized to solve the formulated distributed optimization. The proposed distributed energy management provides microgrids, utility-directly interfaced DERs and responsive demands the opportunity of contributing to better network operational objectives while preserving their privacy and autonomy. Results of numerical simulation using a networked microgrids system consisting of several microgrids, utility directly interfaced DERs and responsive demands validate the soundness and accuracy of the proposed distributed energy management. The proposed method is further tested on a practical two-microgrid system located in Adjuntas, Puerto Rico, and the applicability of the proposed strategy is validated through hardware-in-the-loop (HIL) testing.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Scalable FBP decomposition for cone-beam CT reconstruction

Filtered Back-Projection (FBP) is a fundamental compute intense algorithm used in tomographic image reconstruction. Cone-Beam Computed Tomography (CBCT) devices use a cone-shaped X-ray beam, in comparison to the parallel beam used in older CT generations. Distributed image reconstruction of cone-beam datasets typically relies on dividing batches of images into different nodes. This simple input decomposition, however, introduces limits on input/output sizes and scalability.We propose a novel decomposition scheme and reconstruction algorithm for distributed FPB. This scheme enables arbitrarily large input/output sizes, eliminates the redundancy arising in the end-to-end pipeline and improves the scalability by replacing two communication collectives with only one segmented reduction. Finally, we implement the proposed decomposition scheme in a framework that is useful for all current-generation CT devices (7th gen). In our experiments using up to 1024 GPUs, our framework can construct 40963 volumes, for real-world datasets, in under 16 seconds (including I/O).

Chen, Peng↗
DOE OSTI2021

Identifying and tracking bubbles and drops in simulations: A toolbox for obtaining sizes, lineages, and breakup and coalescence statistics

Knowledge of bubble and drop size distributions in two-phase flows is important for characterizing a wide range of phenomena, including combustor ignition, sonar communication, and cloud formation. The physical mechanisms driving the background flow also drive the time evolution of these distributions. Accurate and robust identification and tracking algorithms for the dispersed phase are necessary to reliably measure this evolution and thereby quantify the underlying mechanisms in interface-resolving flow simulations. The identification of individual bubbles and drops traditionally relies on an algorithm used to identify connected regions. This traditional algorithm can be sensitive to the presence of spurious structures. A cost-effective refinement is proposed to maximize volume accuracy while minimizing the identification of spurious bubbles and drops. An accurate identification scheme is crucial for distinguishing bubble and drop pairs with large size ratios. The identified bubbles and drops need to be tracked in time to obtain breakup and coalescence statistics that characterize the evolution of the size distribution, including breakup and coalescence frequencies, and the probability distributions of parent and child bubble and drop sizes. An algorithm based on mass conservation is proposed to construct bubble and drop lineages using simulation snapshots that are not necessarily from consecutive time steps. These lineages are then used to detect breakup and coalescence events, and obtain the desired statistics. Accurate identification of large-size-ratio bubble and drop pairs enables accurate detection of breakup and coalescence events over a large size range. Accurate detection of successive breakup and coalescence events requires that the snapshot interval be an order of magnitude smaller than the characteristic breakup and coalescence times to capture these successive events while minimizing the identification of repeated confounding events. Together, these algorithms serve as a toolbox for detailed analysis of two-phase simulations, and enable insights into the mechanisms behind bubble and drop formation and evolution in flows of practical importance.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Differentially Private K -Means Clustering Applied to Meter Data Analysis and Synthesis

The proliferation of smart meters has resulted in a large amount of data being generated. It is increasingly apparent that methods are required for allowing a variety of stakeholders to leverage the data in a manner that preserves the privacy of the consumers. The sector is scrambling to define policies, such as the so called ‘15/15 rule’, to respond to the need. However, the current policies fail to adequately guarantee privacy. Here, in this paper, we address the problem of allowing third parties to apply K-means clustering, obtaining customer labels and centroids for a set of load time series by applying the framework of differential privacy. We leverage the method to design an algorithm that generates differentially private synthetic load data consistent with the labeled data. We test our algorithm’s utility by answering summary statistics such as average daily load profiles for a 2-dimensional synthetic dataset and a real-world power load dataset.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Enabling Efficient Sparse Computations using Linear Algebra Aware Compilers

This project developed the LAPIS compiler framework, built on the Multilevel Intermediate Representation (MLIR), to optimize sparse linear algebra operations and support performance portability across diverse architectures. The main innovation of LAPIS is the Kokkos dialect, which allows for lowering codes from a high productivity language to different architectures in an elegant way. The dialect also allows the conversion of lower-level MLIR code to C++ Kokkos code, facilitating the integration of scientific machine learning (SciML) models into applications. To extend LAPIS for distributed memory architectures, a new partition dialect was created to manage the distribution of sparse tensors and express communication patterns for sparse linear algebra operations. This dialect also supports the distributed execution of operators and includes algorithmic optimizations to minimize communication to improve performance. The project also demonstrates that MLIR can enable effective linear algebra-level optimizations, improving performance on different GPUs for both sparse and dense linear algebra kernels. Key applications of LAPIS include sparse linear algebra and graph kernels, TenSQL, a relational database management solution built on GraphBLAS, and the development of subgraph isomorphism and monomorphism kernels, showcasing performance portability. In summary, the LAPIS framework supports productivity, performance, portability, and distributed memory execution, while also enabling linear algebra-level optimizations that are challenging in traditional programming languages, with successful applications ranging from simple sparse linear algebra to complex graph kernels.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Gap-filling eddy covariance methane fluxes: Comparison of machine learning model predictions and uncertainties at FLUXNET-CH4 wetlands

Time series of methane fluxes measured by eddy-covariance require gap-filling to estimate annual emissions. Gap-filling methane fluxes is challenging because of high variability and complex responses to multiple drivers. To date, there is no widely established gap-filling standard for methane, with regards both to the best model algorithms and predictors. In this study, we address the need for standardization by synthesizing results of gap-filling methods applied at 17 wetland sites spanning boreal to tropical regions including all major wetlands classes and two rice paddies. We introduce new procedures for: 1) creating realistic artificial gap scenarios, 2) training and evaluating gap-filling models without overstating performance, and 3) predicting half-hourly methane fluxes and annual emissions with robust uncertainty estimates. We tested a conventional method (marginal distribution sampling) and four machine learning algorithms - penalized linear regression, artificial neural networks, random forests, and boosted decision trees - and four predictor sets, including temporal, meteorological, ecosystem carbon and energy flux, and soil predictors. We find that the conventional method can achieve similar median performance to the machine learning models but is worse than the best machine learning models and relatively insensitive to predictor choices. Of the machine learning models, decision tree algorithms performed the best in cross-validation experiments, even with a baseline predictor set, and artificial neural networks showed comparable performance when using all predictors. Soil temperature was frequently the most important predictor whilst water table depth was important at sites with substantial water table fluctuations, highlighting the value of data on soil conditions. Raw gap-filling uncertainties from the machine learning models were underestimated and we propose a method to calibrate uncertainties to observations. Finally, we gap-fill and provide summary evaluation metrics for all 81 sites in the FLUXNET-CH4 community dataset and publicly release the python code for model development, evaluation, and uncertainty estimation.

42 ENGINEERING↗
DOE OSTI2025

Intercomparison of Three Continuous Monitoring Systems on Operating Oil and Gas Sites

We compare continuous monitoring systems (CMS) from three different vendors on six operating oil and gas sites in the Appalachian Basin using several months of data. We highlight similarities and differences between the three CMS solutions when deployed in the field and compare their output to concurrent top-down aerial measurements and to site-level bottom-up inventories. Furthermore, we compare vendor-provided emission rate estimates to estimates from an open-source quantification algorithm applied to the raw CMS concentration data. This experimental setup allows us to separate the effect of the sensor platform (i.e., sensor type and arrangement) from the quantification algorithm. We find that 1) localization and quantification estimates rarely agree between the three CMS solutions on short time scales (i.e., 30 min), but temporally aggregated emission rate distributions are similar between solutions, 2) differences in emission rate distributions are generally driven by the quantification algorithm, rather than the sensor platform, 3) agreement between CMS and aerial rate estimates varies by CMS solution but is close to parity when CMS estimates are averaged across solutions, and 4) similar sites with similar bottom-up inventories do not necessarily have similar emission characteristics. These results have important implications for developing measurement-informed inventories and for incorporating CMS-inferred emission characteristics into emission mitigation efforts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

An extended numerical manifold method for unsaturated soil-water interaction analysis at micro-scale

To investigate unsaturated soil-water interaction at micro-scale, this work extends the numerical manifold method (NMM) by incorporating a soil-water coupling model considering specific capillary water distribution and capillary force calculation. The soil skeleton is constructed by a soil skeleton generation algorithm with random polygons. To more realistically capture the interaction between soil grains and capillary water, a capillary mechanics-based geometric algorithm is proposed to iteratively calculate the capillary water distribution. The capillary forces corresponding to the capillary water distribution are calculated based on the Young-Laplace equation. The proposed capillary water solving framework is first verified by reproducing the soil-water characteristic curve and the capillary water distribution of an ideal contact-disk model against analytical solutions. To further validate the ability of the capillary water solving framework to predict hydraulic behavior of the real soil, a laboratory test on the Toyoura sand is reproduced numerically. Then an ideal direct shear test is performed to further validate the two-way soil-water coupling procedure, in which a comparison between the numerical and analytical results regarding the shear strength and matric suction is presented. Finally, microscopic hydraulic and compression tests are conducted on two soil specimens with the same porosity and mean grain diameter but different uniformity coefficients. The results elucidate that the extended method is a potential tool to explore unsaturated soil behaviors at micro-scale.

58 GEOSCIENCES↗