SEARCH · Engineering Papers

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Machine Learning-Assisted Distribution System Network Reconfiguration Problem

High penetration from volatile renewable energy resources in the grid and the varying nature of loads raise the need for frequent line switching to ensure the efficient operation of electrical distribution networks. Operators must ensure maximum load delivery, reduced losses, and the operation between voltage limits. However, computations to decide the optimal feeder configuration are often computationally expensive and intractable, making it unfavorable for real-time operations. This is mainly due to the existence of binary variables in the network reconfiguration optimization problem. To tackle this issue, we have devised an approach that leverages machine learning techniques to reshape distribution networks featuring multiple substations. This involves predicting the substation responsible for serving each part of the network. Hence, it leaves simple and more tractable Optimal Power Flow problems to be solved. This method can produce accurate results in a significantly faster time, as demonstrated using the IEEE 37-bus distribution feeder. Compared to the traditional optimization-based approaches, a feasible solution is achieved approximately ten times faster for all the tested scenarios.

deep neural networks↗

DOE OSTI2025

Huge ensembles – Part 2: Properties of a huge ensemble of hindcasts generated with spherical Fourier neural operators

Abstract. In Part 1, we created an ensemble based on spherical Fourier neural operators. As initial condition perturbations, we used bred vectors, and as model perturbations, we used multiple checkpoints trained independently from scratch. Based on diagnostics that assess the ensemble's physical fidelity, our ensemble has comparable performance to operational weather forecasting systems. However, it requires orders-of-magnitude fewer computational resources. Here in Part 2, we generate a huge ensemble (HENS), with 7424 members initialized each day of summer 2023. We enumerate the technical requirements for running huge ensembles at this scale. HENS precisely samples the tails of the forecast distribution and presents a detailed sampling of internal variability. HENS has two primary applications: (1) as a large dataset with which to study the statistics and drivers of extreme weather and (2) as a weather forecasting system. For extreme climate statistics, HENS samples events 4σ away from the ensemble mean. At each grid cell, HENS increases the skill of the most accurate ensemble member and enhances coverage of possible future trajectories. As a weather forecasting model, HENS issues extreme weather forecasts with better uncertainty quantification. It also reduces the probability of outlier events, in which the verification value lies outside the ensemble forecast distribution.

Mahesh, Ankur↗

DOE OSTI2026

Laboratory Evaluation of Federated, Hierarchical Controls for Distribution Power System Management

This presentation provides an overview of the laboratory evaluation of federated, hierarchical controls for distribution power system management.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Modeling Cyber Supply Chain Incidents with Multilayered Graph Motifs

As noted within the literature, supply chain includes people and organizations---manufacturers, integrators, and third-party vendors---that are involved in one or more stages of a product lifecycle. Since supply chains, by definition, include organizations and people, supply chain risk management activities must consider dependencies between an organization's business processes and third-party resources. Just as adversarial tactics can be implemented via techniques implemented via networked computer systems, so can such tactics be expressed via legal business relationships. A cyber incident may have an exponential impact downstream, for example, by leveraging a product's distribution channel (e.g. malicious updates in SolarWinds, buggy updates in CrowdStrike). Similarly, legitimate and legal business relationships also affect the attack surface exposure of systems, enabling long-term persistence and/or unknown impacts to product quality that are hard to detect. This paper catalogs several recent digital supply chain incidents and applies a multilayered network formalism to develop structural indicators (graph motifs) that reflect potentially-adversarial behavior. Finally, we compare and contrast the characteristics of adversarial tactics (e.g. Loss of Availability, Data Collection) that leverage cyber-physical dependencies to those that leverage legal organizational relationships.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Perturbation-Based Diagnosis of False Data Injection Attack Using Distributed Energy Resources

Modern smart grid relies on various sensor measurements for its operational control. In a successful false data injection attack, the attacker manipulates the measurements from the grid sensors such that undetected errors are introduced into the estimates of the system parameters leading to catastrophic situations. This paper proposes a novel perturbation based false data injection attack detection mechanism that utilizes inverter based distributed energy resources (DERs) to create low magnitude perturbation signal in the distribution system voltage that is inconsequential to the normal grid operation. Two voltage sensitivity analysis based algorithms are designed to identify the optimal set of DERs that can create the voltage perturbation signal of desired magnitude. An analytical method of voltage sensitivity analysis is used to compute the magnitude of voltage perturbation signal at each node in a computationally efficient manner. Then, a detection mechanism is developed that checks for the presence of the perturbation sequence in each sensor measurement. A sensor measurement is deemed authentic if the voltage perturbation signal is present in the data. In case of sensor malfunction or cyber-attack, the perturbation signal will not be present in the measurement data. Performance of the proposed attack detection mechanism is validated via simulation of the IEEE 69 bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2023

A high-throughput and low-waste viability assay for microbes

Abstract Counting viable cells is a universal practice in microbiology. The colony-forming unit (CFU) assay has remained the gold standard to measure viability across disciplines, but it is time-intensive and resource-consuming. Here we describe the geometric viability assay (GVA) that replicates CFU measurements over 6 orders of magnitude while reducing over 10-fold the time and consumables required. GVA computes a sample’s viable cell count on the basis of the distribution of embedded colonies growing inside a pipette tip. GVA is compatible with Gram-positive and Gram-negative planktonic bacteria ( Escherichia coli, Pseudomonas aeruginosa and Bacillus subtilis ), biofilms and fungi ( Saccharomyces cerevisiae ). Laborious CFU experiments such as checkerboard assays, treatment time-courses and drug screens against slow-growing cells are simplified by GVA. The ease and low cost of GVA evinces that it can replace existing viability assays and enable viability measurements at previously impractical scales.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2022

Toward a machine-guided approach to energetic material discovery

In this article, we trained a machine learning (ML) model to connect microstructural details of an energetic material formulation to its performance for the purpose of guiding the discovery of new explosive formulations. Our hypothesis was that the algorithm would robustly learn the training data and produce an accurate surrogate model. Specifically, the algorithm learned the relationship between details of the void size distribution (VSD), initiating shock pressure, and the energetic material performance. We used realistic constraints on the VSD and a range of cases were ingested by a physically informed reactive flow model working within a hydrodynamic solver running on high-performance computing resources. The ML algorithm produced a surrogate model that accurately predicted known test points around the parameter space. In addition to the utility of the model and the process used for its development, we noted interesting comparisons between what we, the authors—subject matter experts, would heuristically conclude from the training data and the surrogate model predictions. We detected nuanced details that were missed by the surrogate model; however, these details are not important to an energetic material formulator. We concluded that the algorithm did indeed robustly learn the training data and produce an accurate surrogate model. We further concluded that the surrogate model is a powerful tool to guide the formulator in the absence of subject matter experts and limited-access computing resources.

42 ENGINEERING↗

DOE OSTI2023

Online Model-Free Chance-Constrained Distribution System Voltage Control Using DERs

This paper proposes an online data-driven distributed energy resource management system (DERMS) optimization method using chance-constrained formulation to address distribution system voltage regulation. This is achieved via the local sensitivity factor (LSF)-enabled reformulation of the DER control into a linear programming (LP) problem, which is easy and computationally efficient to solve. The LSF is estimated using online measurements and does not need the assumption of node load information. The latter is usually required for existing optimization-based methods but is difficult to obtain in practice. To mitigate measurement uncertainties, a scenario-based chance-constrained formulation is constructed. Compared with other control methods, the results carried out in a realistic distribution system show that the proposed method can effectively eliminate voltage violation issues.

chance-constrained optimization↗

DOE OSTI2023

Online Model-Free Chance-Constrained Distribution System Voltage Control Using DERs: Preprint

chance-constrained optimization↗

DOE OSTI2021

Computational Math Problems for a Clean Energy Future

Cutting edge computational mathematics are ubiquitous in renewable energy research. Problems in resilient and reliable electric grid operations, infrastructure planning, wind farm yaw control, and more demand sophisticated and scalable computational tools that enable the transition of renewable energy technologies from proof of concept to deployment into our energy system. The mission of the Computational Science Center at NREL is to lead the lab's efforts to solve energy challenges using high-performance computing (HPC), computational science, applied mathematics, scientific data management, visualization, and informatics. In this poster, we provide a short overview of three areas of computational mathematics research at NREL: wind power scenario generation for stochastic grid operations and infrastructure planning, improved rational function approximations for electromagnetic transients codes, and wind farm yaw control using a combination of the Alternating Direction Method of Multipliers (ADMM) and reinforcement learning (RL). Increasing penetrations of renewable energy into power grids motivate the investigation of new approaches to characterizing uncertainty for five-minute economic dispatch problems. Similarly, as the penetration of distributed energy resources on power grids increases, it becomes important to revisit our methods of modelling transient phenomena, i.e. electromagnetic transients programs. Finally, the combination of ADMM and RL for wind farm yaw control presented here can potentially increase the efficiency of the deployed distributed controllers by orders of magnitude.

ADMM↗

DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗

DOE OSTI2022

Data-driven based coordinated smart inverter control for distributed energy resources

Smart inverters (SI) for distributed energy resources (DER) are becoming popular since they have the ability to stabilize as well as restore the voltage and frequency of power systems. Aiming at establishing the mathematical models combined with SI control methods, multiple optimization methods are developed. However, the computational complexity of solving such a mathematical model with various uncertainties limits the real-time application of the SI control. To conquer this challenge, a data-driven-based SI control approach is developed to achieve coordinated control in the high penetration DER system. First, an optimization problem for maximizing the active power generation and minimizing the power loss is designed using the Volt/VAR control. To reduce the time consumption, the recurrent neural network (RNN) is proposed to model the relationship between the uncertainties and control actions during the offline site. The RNN with different sub-structures such as the long short-term memory cell and gated recurrent unit cell are included to enrich the diversity of features. In the last stage, different experiment comparisons, including multiple uncertainties maps and stateof- art machine learning methods, are conducted to verify the effectiveness of the proposed method based on the IEEE 123 bus power system. The results demonstrate that the proposed method can effectively achieve a rapid and coordinated control with a lower error rate.

Qiu, Wei↗

DOE OSTI2024

Equity and modeling in sustainability science: Examples and opportunities throughout the process

Equity is core to sustainability, but current interventions to enhance sustainability often fall short in adequately addressing this linkage. Models are important tools for informing action, and their development and use present opportunities to center equity in process and outcomes. This Perspective highlights progress in integrating equity into systems modeling in sustainability science, as well as key challenges, tensions, and future directions. We present a conceptual framework for equity in systems modeling, focused on its distributional, procedural, and recognitional dimensions. We discuss examples of how modelers engage with these different dimensions throughout the modeling process and from across a range of modeling approaches and topics, including water resources, energy systems, air quality, and conservation. Synthesizing across these examples, we identify significant advances in enhancing procedural and recognitional equity by reframing models as tools to explore pluralism in worldviews and knowledge systems; enabling models to better represent distributional inequity through new computational techniques and data sources; investigating the dynamics that can drive inequities by linking different modeling approaches; and developing more nuanced metrics for assessing equity outcomes. We also identify important future directions, such as an increased focus on using models to identify pathways to transform underlying conditions that lead to inequities and move toward desired futures. By looking at examples across the diverse fields within sustainability science, we argue that there are valuable opportunities for mutual learning on how to use models more effectively as tools to support sustainable and equitable futures.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

The Edge of Exploration: An Edge Storage and Computing Framework for Ambient Noise Seismic Interferometry Using Internet of Things Based Sensor Networks

Recent technological advances have reduced the complexity and cost of developing sensor networks for remote environmental monitoring. However, the challenges of acquiring, transmitting, storing, and processing remote environmental data remain significant. The transmission of large volumes of sensor data to a centralized location (i.e., the cloud) burdens network resources, introduces latency and jitter, and can ultimately impact user experience. Edge computing has emerged as a paradigm in which substantial storage and computing resources are located at the “edge” of the network. In this paper, we present an edge storage and computing framework leveraging commercially available components organized in a tiered architecture and arranged in a hub-and-spoke topology. The framework includes a popular distributed database to support the acquisition, transmission, storage, and processing of Internet-of-Things-based sensor network data in a field setting. We present details regarding the architecture, distributed database, embedded systems, and topology used to implement an edge-based solution. Lastly, a real-world case study (i.e., seismic) is presented that leverages the edge storage and computing framework to acquire, transmit, store, and process millions of samples of data per hour.

58 GEOSCIENCES↗

DOE OSTI2025

LandScan HD: a high-resolution gridded ambient population methodology for the world

Unwarned population distributions accounting for routine human activities are needed to address many global human security challenges, including disasters, conflict, and infrastructure demand. LandScan High Definition (LSHD) supports this need through gridded ambient population estimates that measure average human presence between daytime and nighttime at a high spatial resolution of 3 arcseconds (approximately 90 m). Although LSHD has traditionally been produced on a country-specific basis, advances in global foundational data and computational resources now enable scaling its methodology to the world. Combining aspects of top-down and bottom-up gridded population methods, LSHD allocates subnational population totals from authoritative statistics to built-up areas based on occupancy estimates for multiple facility types (e.g., residential, commercial) and then reaggregates these estimates to a global population grid. We scale this approach by organizing the LSHD data stack into a 1° resolution tileset of vector analytic features, enabling an efficient and repeatable workflow for all countries worldwide. Examining the Philippines as an output of the global LSHD baseline dataset, we contrast unwarned and residential (WorldPop) population distributions by (1) exploring a practical application of flood risk assessment and (2) evaluating their congruence with outcomes of collective human activities (subnational CO 2 emissions). Finally, we discuss plans to address current LSHD limitations through data/modeling and uncertainty quantification improvements and provide outlook for workflow automation and extending the model to social, demographic and economic population characteristics.

Building morphology↗

DOE OSTI2023

HEPnOS: a Specialized Data Service for High Energy Physics Analysis

In this paper, we present HEPnOS, a distributed data service for managing data produced by high-energy physics (HEP) experiments. Using HEPnOS, HEP applications can use HPC resources more effciently than traditional fle-based applications. The fle-based model leads to a rigid, chunk-based allocation of computational resources and limits the number of cores that can be used concurrently by an HEP application. The fundamental problem is that organizing domain-specifc data into fles inadvertently introduces a single, artifcial, confated tuning parameter that puts key optimization goals into confict: larger fle sizes reduce metadata overhead and thus improve I/O effciency, but smaller fle sizes provide more opportunity for workfow parallelism and load balancing. In this work, we introduce a domain-specifc data service that decouples that constraint so that data can be accessed and processed in its natural granularity while still maintaining I/O effciency. By removing the constraints introduced by fle handling we are able to obtain better scaling and make effcient use of more cores for processing a fxed-sized data sample. We demonstrate the improved scalability by using an application developed in the fle-based paradigm and comparing it to a version modifed to use HEPnOS.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Permutationally Invariant Polynomial Expansions with Unrestricted Complexity

A general strategy is presented for constructing and validating permutationally invariant polynomial (PIP) expansions for chemical systems of any stoichiometry. Demonstrations are made for three categories of gas-phase dynamics and kinetics: collisional energy-transfer trajectories for predicting pressure-dependent kinetics, three-body collisions for describing transient van der Waals adducts relevant to atmospheric chemistry, and nonthermal reactivity via quasiclassical trajectories. In total, 30 systems are considered with up to 15 atoms and 39 degrees of freedom. Permutational invariance is enforced in PIP expansions with as many as 13 million terms and 13 permutationally distinct atom types by taking advantage of petascale computational resources. The quality of the PIP expansions is demonstrated through the systematic convergence of in-sample and out-of-sample errors with respect to both the number of training data and the order of the expansion, and these errors are shown to predict errors in the dynamics for both reactive and nonreactive applications. Here, the parallelized code distributed as part of this work enables the automation of PIP generation for complex systems with multiple channels and flexible user-defined symmetry constraints and for automatically removing unphysical unconnected terms from the basis set expansions, all of which are required for simulating complex reactive systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Enhancing approximate modular Bayesian inference by emulating the conditional posterior

In modular Bayesian analyses, complex models are composed of distinct modules, each representing different aspects of the data or prior information. In this context, fully Bayesian approaches can sometimes lead to undesirable feedback between modules, compromising the integrity of the inference. The “cut-distribution” prevents unwanted influence between modules by “cutting” feedback. The direct sampling (DS) algorithm is standard practice for approximating the cut-distribution, but it can be computationally intensive, especially when the number of imputations required is large. An enhanced method is proposed, the Emulating the Conditional Posterior (ECP) algorithm, which leverages emulation to increase the number of imputations. Through numerical experiment it is demonstrated that the ECP algorithm outperforms the traditional DS approach in terms of accuracy and computational efficiency, particularly when resources are constrained. Here, it is also shown how the DS algorithm can be improved using ideas from design of experiments. Some practical recommendations are given for algorithm choice in modular Bayesian analyses.

97 MATHEMATICS AND COMPUTING↗