Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49
All sourcesNASA NTRSDOE OSTI
NASA NTRS1994

Optimal pattern distributions in Rete-based production systems

Since its introduction into the AI community in the early 1980's, the Rete algorithm has been widely used. This algorithm has formed the basis for many AI tools, including NASA's CLIPS. One drawback of Rete-based implementation, however, is that the network structures used internally by the Rete algorithm make it sensitive to the arrangement of individual patterns within rules. Thus while rules may be more or less arbitrarily placed within source files, the distribution of individual patterns within these rules can significantly affect the overall system performance. Some heuristics have been proposed to optimize pattern placement, however, these suggestions can be conflicting. This paper describes a systematic effort to measure the effect of pattern distribution on production system performance. An overview of the Rete algorithm is presented to provide context. A description of the methods used to explore the pattern ordering problem area are presented, using internal production system metrics such as the number of partial matches, and coarse-grained operating system data such as memory usage and time. The results of this study should be of interest to those developing and optimizing software for Rete-based production systems.

Scott, Stephen L.↗
NASA NTRSDate not supplied

Shock-Layer Radiation Insights Available Through Flowfield Property Binning

This paper introduces a method for assessing similarity between available shock-layer radiation measurements and the radiative heating to a vehicle of interest. This method will be referred to as the flowfield property binning (FPB) approach. The challenge in developing this approach is that radiometer measurements from a surface normal line of sight, observed radiation measurements of the entire shock-layer from a distance, or measured shock-tube nonequilibrium profiles are notably different geometrically than the radiative flux to a vehicle surface. Therefore, the relative contributions of various regions of the shock-layer to the measurement, and how these relative contributions compare with the radiative heating to a vehicle of interest, are unclear. By tracking the radiation contribution from each flowfield location, and then grouping this contribution based on two local flowfield properties, these geometric differences are removed. This provide a two dimensional group, or bin, distribution that represents the radiation environment in the same format for various types of radiation simulations, such as observed radiation or radiative flux to a vehicle surface. In the limits of optically-thin radiation transport and thermochemical equilibrium flow, the choice of pressure and temperature as the binning flowfield properties is shown to provide a bin distribution that uniquely defines the radiation simulation. This concept is found to be sufficient for establishing similarity between two cases even when the optically-thin radiation transport and thermochemical equilibrium assumptions are not satisfied. A simple metric of comparison between bin distributions from different cases is provided by the histogram intersection algorithm. This algorithm results in a histogram intersection factor (H) between 0 and 1, which represents the fraction of the radiation that is similar between the two cases. Therefore, if H evaluated between bin distributions for two completely different radiation problems is near 1, then they represent essentially the same radiation problem. This means that they depend on the same weighting of various radiation modeling parameters evaluated at the same pressure and temperature. If one of these cases agrees well with a radiation measurement, then the matching bin distributions provide justification for applying this experimental validation to the other case. Even for H values as low as 0.2, the measurement case could be considered relevant if there are no other useful measurements. This approach allows observed radiation measurements to directly inform radiative heating simulations, which allows NASA’s past and future investments in observed radiation measurements to directly inform the radiative heating uncertainty or margins for future vehicle design. This FPB approach is also used to determine shock-tube conditions for informing backshell radiative heating simulations, where the normal shock velocity and Lagrangian time from the shock is used for the binning variables instead of pressure and temperature.

Shock Layer Radiation↗
NASA NTRS1999

In Pursuit of Abundance Anomalies and the FIP Effect in Late-Type Stellar Coronae

Key EUVE and ASCA data have been retrieved from their respective archives. New software has been written in the IDL language to carry out data analysis and to interface with the relevant atomic physics databases. During the analysis of ASCA spectra, it was found that the abundances of elements other than Fe could not be constrained very well, and Fe abundances were not constrained unless the underlying emission measure distribution model was reasonably well-determined. Consequently, the study has concentrated on the quantity Fe/H. A method has been developed as a means of deriving Fe/H based on fitting the continuum to EUVE spectra, thereby using the Fe lines to determine the Fe abundance. A Monte Carlo Markov Chain algorithm was also developed to determine the emission measure distribution based on observed spectral lines. This is the first application of this type of monte carlo approach to this scientific problem. This work has resulted in three scientific publications, one of which published, one of which is now ready for submission and the other of which is still in preparation:

Drake, Jeremy↗
DOE OSTI2025

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗
NASA NTRS2005

A Comparative Study of Probability Collectives Based Multi-agent Systems and Genetic Algorithms

We compare Genetic Algorithms (GA's) with Probability Collectives (PC), a new framework for distributed optimization and control. In contrast to GA's, PC-based methods do not update populations of solutions. Instead they update an explicitly parameterized probability distribution p over the space of solutions. That updating of p arises as the optimization of a functional of p. The functional is chosen so that any p that optimizes it should be p peaked about good solutions. The PC approach works in both continuous and discrete problems. It does not suffer from the resolution limitation of the finite bit length encoding of parameters into GA alleles. It also has deep connections with both game theory and statistical physics. We review the PC approach using its motivation as the information theoretic formulation of bounded rationality for multi-agent systems. It is then compared with GA's on a diverse set of problems. To handle high dimensional surfaces, in the PC method investigated here p is restricted to a product distribution. Each distribution in that product is controlled by a separate agent. The test functions were selected for their difficulty using either traditional gradient descent or genetic algorithms. On those functions the PC-based approach significantly outperforms traditional GA's in both rate of descent, trapping in false minima, and long term optimization.

Huang, Chien-Feng↗
NASA NTRS2009

Trajectory-Oriented Approach to Managing Traffic Complexity: Trajectory Flexibility Metrics and Algorithms and Preliminary Complexity Impact Assessment

This document describes exploratory research on a distributed, trajectory oriented approach for traffic complexity management. The approach is to manage traffic complexity based on preserving trajectory flexibility and minimizing constraints. In particular, the document presents metrics for trajectory flexibility; a method for estimating these metrics based on discrete time and degree of freedom assumptions; a planning algorithm using these metrics to preserve flexibility; and preliminary experiments testing the impact of preserving trajectory flexibility on traffic complexity. The document also describes an early demonstration capability of the trajectory flexibility preservation function in the NASA Autonomous Operations Planner (AOP) platform.

Idris, Husni↗
DOE OSTI2024

A remote sensing algorithm for vertically resolved cloud condensation nuclei number concentrations from airborne and spaceborne lidar observations

Cloud condensation nuclei (CCN) are mediators of aerosol–cloud interactions (ACIs), contributing to the largest uncertainties in the understandings of global climate change. We present a novel remote-sensing-based algorithm that quantifies the vertically resolved CCN number concentrations (N CCN ) using aerosol optical properties measured by a multiwavelength lidar. The algorithm considers five distinct aerosol subtypes with bimodal size distributions. The inversion used the lookup tables developed in this study, based on the observations from the Aerosol Robotic Network, to efficiently retrieve optimal particle size distributions from lidar measurements. The method derives dry aerosol optical properties by implementing hygroscopic enhancement factors in lidar measurements. The retrieved optically equivalent particle size distributions and aerosol-type-dependent particle composition are utilized to calculate critical diameters using κ-Köhler theory and N CCN at six supersaturations ranging from 0.07 % to 1.0 %. Sensitivity analyses indicate that uncertainties in extinction coefficients and relative humidity greatly influence the retrieval error in N CCN . The potential of this algorithm is further evaluated by retrieving N CCN using airborne lidar from the NASA ObseRvations of Aerosols above CLouds and their intEractionS (ORACLES) campaign and is validated against simultaneous measurements from the CCN counter. The independent validation with robust correlation demonstrates promising results. Furthermore, the N CCN has been retrieved for the first time using a proposed algorithm from spaceborne lidar – Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) – measurements. The application of this new capability demonstrates the potential for constructing a 3D CCN climatology at a global scale, which helps to better quantify ACI effects and thus reduce the uncertainty in aerosol climate forcing.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1998

Parallel Implicit Algorithms for CFD

The main goal of this project was efficient distributed parallel and workstation cluster implementations of Newton-Krylov-Schwarz (NKS) solvers for implicit Computational Fluid Dynamics (CFD.) "Newton" refers to a quadratically convergent nonlinear iteration using gradient information based on the true residual, "Krylov" to an inner linear iteration that accesses the Jacobian matrix only through highly parallelizable sparse matrix-vector products, and "Schwarz" to a domain decomposition form of preconditioning the inner Krylov iterations with primarily neighbor-only exchange of data between the processors. Prior experience has established that Newton-Krylov methods are competitive solvers in the CFD context and that Krylov-Schwarz methods port well to distributed memory computers. The combination of the techniques into Newton-Krylov-Schwarz was implemented on 2D and 3D unstructured Euler codes on the parallel testbeds that used to be at LaRC and on several other parallel computers operated by other agencies or made available by the vendors. Early implementations were made directly in Massively Parallel Integration (MPI) with parallel solvers we adapted from legacy NASA codes and enhanced for full NKS functionality. Later implementations were made in the framework of the PETSC library from Argonne National Laboratory, which now includes pseudo-transient continuation Newton-Krylov-Schwarz solver capability (as a result of demands we made upon PETSC during our early porting experiences). A secondary project pursued with funding from this contract was parallel implicit solvers in acoustics, specifically in the Helmholtz formulation. A 2D acoustic inverse problem has been solved in parallel within the PETSC framework.

Keyes, David E.↗
NASA NTRS2018

Observing the Full Spectrum of the Rain Drop Size Distribution

GPM DSD retrievals exhibit inconsistencies between GV, DPR and Combined algorithm retrievals. Development of positive bias in convective Dm rain DSD noted, and strongest in KuPR retrieval. Associated epsilons are too low and result in markedly reduced convective rain rates (a current issue in the retrievals). Source may be NUBF. Issues with the large end of the DSD not withstanding, on the small end of the DSD, combined MPS and 2DVD measurements fit with generalized gamma functions exhibit strong potential for representing the entire spectrum of the DSD and subsequently the whole rain rate spectrum.

Hydrology↗
DOE OSTI2023

Hardware-in-the-Loop Evaluation of Grid-Edge DER Chip Integration Into Next-Generation Smart Meters

To facilitate the implementation of distributed energy resource management systems (DERMS), we propose to insert a grid-edge distributed energy resource (DER) chip hosting a DERMS algorithm into the next generation of smart meters. This will create a pathway for the wide adoption of DERMS technology because many utilities plan to invest in advanced metering infrastructure in the near future. This will also bridge the gap between an electrical power utility and DERs behind the meter. The DER chip is designed to follow power direction signals from the DERMS coordinator while balancing its local objectives. We tested the chip using a controller- and power-hardware-in-the-loop evaluation under three scenarios that a DERMS could face in the real world. The DER chip was capable of and effective at directing four heterogeneous DERs to respond to a DERMS coordinator for grid services (e.g., voltage regulation and a virtual power plant).

distributed energy resource management system↗
DOE OSTI2023

Hardware-in-the-Loop Evaluation of Grid-Edge DER Chip Integration Into Next-Generation Smart Meters: Preprint

To facilitate the implementation of distributed energy resource management systems (DERMS), we propose to insert a grid-edge distributed energy resource (DER) chip hosting a DERMS algorithm into the next generation of smart meters. This will create a pathway for the wide adoption of DERMS technology because many utilities plan to invest in advanced metering infrastructure in the near future. This will also bridge the gap between an electrical power utility and DERs behind the meter. The DER chip is designed to follow power direction signals from the DERMS coordinator while balancing its local objectives. We tested the chip using a controller- and power-hardware-in-the-loop evaluation under three scenarios that a DERMS could face in the real world. The DER chip was capable of and effective at directing four heterogeneous DERs to respond to a DERMS coordinator for grid services (e.g., voltage regulation and a virtual power plant).

distributed energy resource management system↗
DOE OSTI2025

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗
NASA NTRS1999

Solar Microflare with BATSE

Our work on detecting and cataloging solar microflares using an automated method is illustrated in the accompanying figure. The figure represents the solar microflare distribution during the period of April 1991 to November 1992, the height of solar activity after the launch of The Compton Gamma Ray Observatory (CGRO). It also shows the distribution extending below the distribution obtained at Goddard Space Flight Center (GSFC) by manual means. We have implemented significant refinements in the search algorithm. The algorithm in its simplest form searches for transient events and based upon the distribution of the signal among the different Burst and Transient Source Experiment (BATSE) detectors, we can assign it to be of solar origin if the signal distribution conforms to what one expects from a burst or transient from that direction. One of the major problems in the earlier effort was to search for microflares and large flares simultaneously. The requirement for a dynamic range of almost 10 (exp 4) resulted in ambiguous identifications at the low side of the distribution. We have since restricted the search to events with peak count rates under 2000 s (exp -1). Larger events are easily identified in the manual search, so we have chosen not to duplicate that work. The second problem was that missing counts existed below channel 0 in the Burst and Transient Source Experiment Large Area Detector data (BATSE LAD). These have been recovered and are now included in the search process. This provides data below 20 keV, and as we get closer to the thermal part of the spectrum, it provides greater sensitivity. The third problem was that too many BATSE detector were used in the search. Detectors with pointing directions far from the Sun, although detecting the event, had poorly known responses. Detectors greater than approximately 60 deg. off the Sun are no longer included in the search process. By reducing the systematic errors with the large off-axis detectors we can conduct more rigorous statistical tests of a candidate event to ascertain whether it originated from the solar direction. We have reprocessed the period in the early mission that covers solar maximum and constructed the microflare distribution shown in the figure. The results of the automated search start to deviate from the manual search results below about 1000 s (exp -1). Not only do we now have this distribution but we have a database of solar microflares that was used to construct the distribution. This database contains the signal at higher energy channels as well as that in channel zero (and below). From this one can, using software at GSFC, construct a photon spectrum for some of the larger microflares. It can also be used in other solar studies, especially those that correlate the X-ray flux with emission at other wavelengths. With some additional effort we hope to integrate this database into the corresponding one residing at the Solar Data Analysis Center at GSFC. The entire CGRO mission's data can now be reprocessed to obtain the microflare distribution at all phases of the solar cycle. This work is in progress. The results of this work will be presented in forthcoming scientific workshops and conferences.

Ryan, James M.↗
DOE OSTI2023

Digital encoding algorithm for pixelated detectors

A detector for imaging and efficiently digitizing a spatial distribution of photon flux includes pixel circuits that compressively encode pixel values generated by integrated analog to digital converters (ADCs). On-pixel digital compression circuits (DCCs) implement compression to increase continuous frame rate by reducing the number of bits per pixel while keeping quantization error below Poisson noise. Several mapping algorithms for photon-counting and charge-integrating detectors and compact digital logic implementations are presented.

Hammer, Michael P.↗
DOE OSTI2022

IMoFi - Intelligent Model Fidelity: Physics-Based Data-Driven Grid Modeling to Accelerate Accurate PV Integration (Final Report)

This report summarizes the work performed under a project funded by U.S. DOE Solar Energy Technologies Office (SETO) to use grid edge measurements to calibrate distribution system models for improved planning and grid integration of solar PV. Several physics-based data-driven algorithms are developed to identify inaccuracies in models and to bring increased visibility into distribution system planning. This includes phase identification, secondary system topology and parameter estimation, meter-to-transformer pairing, medium-voltage reconfiguration detection, determination of regulator and capacitor settings, PV system detection, PV parameter and setting estimation, PV dynamic models, and improved load modeling. Each of the algorithms is tested using simulation data and demonstrated on real feeders with our utility partners. The final algorithms demonstrate the potential for future planning and operations of the electric power grid to be more automated and data-driven, with more granularity, higher accuracy, and more comprehensive visibility into the system.

14 SOLAR ENERGY↗
DOE OSTI2022

IMoFi (Intelligent Model Fidelity): Physics-Based Data-Driven Grid Modeling to Accelerate Accurate PV Integration Updated Accomplishments

This report summarizes the work performed under a project funded by U.S. DOE Solar Energy Technologies Office (SETO), including some updates from the previous report SAND2022-0215, to use grid edge measurements to calibrate distribution system models for improved planning and grid integration of solar PV. Several physics-based data-driven algorithms are developed to identify inaccuracies in models and to bring increased visibility into distribution system planning. This includes phase identification, secondary system topology and parameter estimation, meter-to-transformer pairing, medium-voltage reconfiguration detection, determination of regulator and capacitor settings, PV system detection, PV parameter and setting estimation, PV dynamic models, and improved load modeling. Each of the algorithms is tested using simulation data and demonstrated on real feeders with our utility partners. The final algorithms demonstrate the potential for future planning and operations of the electric power grid to be more automated and data-driven, with more granularity, higher accuracy, and more comprehensive visibility into the system.

14 SOLAR ENERGY↗
DOE OSTI2022

Rain Drop Size Distributions Estimated from NOAA Snow-Level Radar Data

Using NOAA’s S-band High-Power Snow-Level Radar (HPSLR), a technique for estimating the rain drop size distribution (DSD) above the radar is presented. This technique assumes the DSD can be described by a four parameter, generalized gamma distribution (GGD). Using the radar’s measured average Doppler velocity spectrum and a value (assumed, measured, or estimated) of the vertical air motion w, an estimate of the GGD is obtained. Four different methods can be used to obtain w. One method that estimates a mean mass-weighted raindrop diameter D m from the measured reflectivity Z produces realistic DSDs compared to prior literature examples. These estimated DSDs provide evidence that the radar can retrieve the smaller drop sizes constituting the “drizzle” mode part of the DSD. Here, this estimation technique was applied to 19 h of observations from Hankins, North Carolina. Results support the concept that DSDs can be modeled using GGDs with a limited range of parameters. Further work is needed to validate the described technique for estimating DSDs in more varied precipitation types and to verify the vertical air motion estimates.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Situational awareness-enhancing community-level load mapping with opportunistic machine learning

Motivated by present and forthcoming challenges in the adoption and integration of distributed renewable energy, we develop a machine learning (ML) approach that builds short-fuse mappings connecting the occasionally-unobservable true load in one target community with information-rich signals collected from relatively more instrumented reference communities. Our setting is inspired by and tailored to target communities with significant unobservable behind-the-meter solar generation, where true load (a relatively well-behaved quantity of interest to grid operators) is hard to discern during daytime due to insufficient instrumentation and/or privacy reasons, but that can be related to reference communities with low unobservable distributed variable generation or with sufficient instrumentation. The developed mapping, herein realized with Support Vector Machine regression, is built using nighttime data from all communities, when their distributed generation is low or zero. Our ML algorithm opportunistically learns to correlate signals of interest and then is operationally used the next day to shed light into target community load evolution. The mapping is subsequently rebuilt, rolling its short-fuse scope perpetually forward in time. Here, we demonstrate the efficacy of our approach on nine synthetically generated topologies and associated timeseries stemming from real-world data, on which we observe cumulative error performance that yields lower than 10% and 15% daily-averaged mean absolute percentage errors in target community load estimation on more than about 75% and 90% of days, respectively, in multiple yearly evaluations that shed light on long-term performance also under seasonal and one-off effects. The proposed ML-powered methodology can offer grid operators much-improved visibility into a previously obscure space and can also serve as an additional source of information in broader, multi-modal solar disaggregation solutions.

14 SOLAR ENERGY↗