Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Impact of FERC Order 2222 on DER Participation Rules in US Electricity Markets

Electricity markets in the bulk grid are beginning to implement market mechanisms that support the procurement of flexible capabilities from wide range of technologies, including distributed energy resources (DERs). The flexibility of these resources will help counterbalance supply uncertainties from large-scale integration of variable renewable generation. To encourage development of distributed and aggregated market participants, FERC Order 2222 was issued in September 2020 to require each Independent System Operator (ISO) in the US to implement rules that enable broader participation from aggregations of DERs in the bulk market. The following paper first describes the generic design of ISO markets before introducing the new market participation rules that ISOs have proposed for compliance with Order 2222. The paper then describes how software performance issues may continue to affect the eligibility requirements and offer structures for DER aggregations participating in ISOs, noting that continued research on computational methods may help reduce burdens for DER integration. The prospects for transmission and distribution system coordination is second major issue discussed, which will require minor changes to existing processes in the short term. In the longer term, there is more opportunity for more wide-ranging reforms, such as the development of a Distribution System Operator (DSO) framework. Newly proposed market rules may affect how Transactive Energy Systems (TES) will help facilitate efficient formation of DER aggregations and operation of the individual DERs within an aggregation. Within the TES context, the challenge is to fully understand how resource eligibility and operational and planning coordination methods will affect the design and implementation of TES.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

The ATLAS experiment software on ARM

With an increased dataset obtained during the Run 3 of the LHC at CERN and the even larger expected increase of the dataset by more than one order of magnitude for the HL-LHC, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM architecture is becoming a competitive and energy efficient alternative. Some surveys indicate its increased presence in HPCs and commercial clouds, and some WLCG sites have expressed their interest. Chip makers are also developing their next generation solutions on ARM architectures, sometimes combining ARM and GPU processors in the same chip. Consequently it is important that the ATLAS software embraces the change and is able to successfully exploit this architecture. We report on the successful porting to ARM of the Athena software framework, which is used by ATLAS for both online and offline computing operations. Furthermore we report on the successful validation of simulation workflows running on ARM resources. For this we have set up an ATLAS Grid site using ARM compatible middleware and containers on Amazon Web Services (AWS) ARM resources. The ARM version of Athena is fully integrated in the regular software build system and distributed in the same way as other software releases. In addition, the workflows have been integrated into the HEPscore benchmark suite which is the planned WLCG wide replacement of the HepSpec06 benchmark used for Grid site pledges. In the overall porting process we have used resources on AWS, Google Cloud Platform (GCP) and CERN. A performance comparison of different architectures and resources will be discussed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Measurement and applications: Exploring the challenges and opportunities of hierarchical federated learning in sensor applications

Sensor applications have become ubiquitous in modern society as the digital age continues to advance. AI-based techniques (e.g., machine learning) are effective at extracting actionable information from large amounts of data. An example would be an automated water irrigation system that uses AI-based techniques on soil quality data to decide how to best distribute water. However, these AI-based techniques are costly in terms of hardware resources, and Internet-of-Things (IoT) sensors are resource-constrained with respect to processing power, energy, and storage capacity. These limitations can compromise the security, performance, and reliability of sensor-driven applications. To address these concerns, cloud computing services can be used by sensor applications for data storage and processing. Unfortunately, cloud-based sensor applications that require real-time processing, such as medical applications (e.g., fall detection and stroke prediction), are vulnerable to issues such as network latency due to the sparse and unreliable networks between the sensor nodes and the cloud server [1]. As users approach the edge of the communications network, latency issues become more severe and frequent. A promising alternative is edge computing, which provides cloud-like capabilities at the edge of the network by pushing storage and processing capabilities from centralized nodes to edge devices that are closer to where the data are gathered, resulting in reduced network delays [2], [3].

Po-Leen Ooi, Melanie↗
DOE OSTI2022

Emulation Framework for Distributed Large-Scale Systems Integration

Recent trends in systems engineering include integration of very large-scale systems, which entails significant challenges when they are geographically dispersed. In these scenarios, intelligent integration of distributed large-scale systems requires significant coordination among hardware elements as well as all software components. The approach of integrated systems (both computing platform and experimental equipment) for end-to-end orchestration is called federation. Virtual frameworks can aid in the testing, assessment, and implementation of a functional system of interconnected resources. We present an emulation framework that replicates the software environments of multi-site federations of computing systems and instruments. Our emulation framework allows systems engineers to reduce developmentcost and avoid disruptions to production infrastructure. Our framework was effectively used to develop and test software modules for various tasks including container orchestration and instrument access. For performance assessment, however, the emulated framework is severely limited in providing accurate network and IO measurements at 10 Gbps and higher data rates. The data transfer performance profiles estimated using these emulated measurements are usually inaccurate for high bandwidth and high latency connections, since emulation does not accurately reflect the critical network transport dynamics.We utilize measurements from a physical testbed with hardware network emulators to obtain data transfer profiles that closely match the expected profiles for the emulated federations. We show the effectiveness of our approach by an illustrative example of integrated (federated) multi-site ultra large-scale systems that are connected via high speed wide area networks.

Imam, Neena↗
DOE OSTI2022

Stochastic Strategic Participation of Active Distribution Networks With High-Penetration DERs in Wholesale Electricity Markets

With the increasing penetration of distributed energy resources (DERs), traditional distribution networks as load-serving entities in wholesale electricity markets, now evolve towards active distribution networks (ADNs) which can proactively participate in wholesale markets by optimally controlling the DERs in their networks. A stochastic bilevel optimization model is proposed in this paper for the strategic participation of ADNs and DERs to provide energy and grid services in wholesale electricity markets. The bilevel optimization model can capture the interactions between the ADN and the wholesale energy and ancillary service markets, considering the uncertainties of DERs in the ADN. In the upper-level model, the ADN makes optimal decisions on energy and reserve bidding considering the availability, uncertainties, and flexibility of DERs. The joint energy and reserve market-clearing of the independent system operator (ISO) is modeled as the lower-level problem. Using strong duality theory and Karush-Kuhn Tucker (KKT) conditions, the proposed bilevel optimization problem is reformulated as mathematical programming with equilibrium constraints (MPEC) problem and further converted into a computationally-solvable mixed-integer second-order-cone programming (MISOCP) model. The simulation results demonstrate the effectiveness of the model and the interactions between an ADN and wholesale electricity markets.

active distribution network↗
DOE OSTI2020

Harnessing the power of ab initio calculations, distributed computing and machine learning to efficiently locate extreme molecules for use in carbon-based solar cells (Final Technical Report)

The use of high-throughput virtual screening (HTVS) tools is a powerful tool to expedite the materials discovery of commercially relevant materials. In previous years, our group has developed a molecular discovery platform to generate libraries in order to obtain suitable candidates for different applications, starting from the Harvard Clean Energy Project. This platform is suitable to test in-silico on traditional supercomputing clusters and shared resources, for example, in the IBM World Community Grid. In this project, we used the molecular discovery platform to create and screen a library of candidates of organic photovoltaic (OPVs) molecules. Based on a set of candidates created with combinations of molecular moieties, we were able to filter, by conformation stability, the energy of electronic orbitals and approximated power conversion efficiencies (PCE). To improve the predictions of orbital energies calculated and the PCEs, we used Gaussian Process regression and two sets of molecules. These sets correspond to electronic structure calculations of a higher level of theory and experimental PCE values, respectively. Finally, we selected a subset of the best candidates (molecules with a PCE higher than 10%) to understand its absorbance properties with TD-DFT. This project has demonstrated the capabilities of our molecular discovery platform for HTVS. Finally, machine learning can help us to introduce more complex effects included in bulk conditions and computational intensive calculations on models.

14 SOLAR ENERGY↗
DOE OSTI2020

Verification of an improved equation-free projective integration method for neoclassical plasma-profile evolution in tokamak geometry

A brute-force, long-time gyrokinetic simulation of plasma profile evolution in magnetic fusion devices is not desirable due to large computational resource requirements and a possible accumulation of numerical error. The equation-free projective integration method of Keverekidis et al. [Commun. Math. Sci. 1(4), 715–762 (2003)] is one of the outstanding candidates in projecting micro-scale simulations to a longer timescale. However, its application to tokamak plasma has not been fruitful due to the appearance of spurious transient oscillations in the lifting process, which are present when the kinetic simulations are initialized with a simplified model distribution function and which make the kinetic simulations to deviate from the desired paths. In this work, a kinetically informed lifting algorithm is added to the equation-free projective integration method, which is then verified in the electrostatic gyrokinetic particle-in-cell code XGCa [R. Hager and C. S. Chang, Phys. Plasmas 23, 042503 (2016)] for a neoclassical ion heat transport problem with adiabatic electrons. This new lifting operator is demonstrated to control spurious transients, enabling an over four-times reduction in the overall computing time in the time-evolution of the ion temperature profile in an axisymmetric toroidal plasma. Further reduction in the computing time is found to be limited due to the stability properties of the linear least squares projective integrator.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Reflections on the Shifting Experiences of Scientific Infrastructure

Infrastructure of all types is fundamental to modern work and life. Computing for scientific work, especially, extends from distributed local research sites, often at the edges of other major systems, outward into globally connected high-performance facilities and infrastructures. This commentary reviews longstanding research on the social characteristics of infrastructure. We reflect on social concerns that affect the ongoing development, use, and maintenance of a wide range of scientific computing and data resources. Reflecting on the social nature of infrastructure is timely for Computing in Science & Engineering readers, given continued emphasis on developing even more expansive platforms for data and artificial intelligence work in science (e.g., the United States’ Genesis Mission). We assert that, regardless of technological advances, the complex nature of scientific research and data will require continued understanding of longstanding and nascent social practices across varied communities. This is fundamentally necessary to build and sustain usable infrastructure or platforms that can productively advance scientific research.

Paine, Drew [Lawrence Berkeley National Laboratory↗
DOE OSTI2021

Residential Demand Side Aggregation of Privacy-Conscious Consumers

The increasing adoption of smart meters has led to growing concerns regarding privacy risks stemming from the high resolution measurements. This has given rise to privacy protection techniques that physically alter the consumer's energy load profile, masking private information by using localised devices, e.g. batteries or flexible loads. Meanwhile, there has also been increasing interest in aggregating the distributed energy resources (DERs) of residential consumers to provide services to the grid. In this paper, we propose an online distributed algorithm to aggregate the DERs of privacy-conscious consumers to provide services to the grid, whilst preserving their privacy. Results show that the optimisation solution from the distributed method converges to one close to the optimum computed using an ideal centralised solution method, balancing between grid service provision, consumer preferences and privacy protection. More importantly, the distributed method preserves consumer privacy, and does not require high-bandwidth two-way communications infrastructure.

ancillary services↗
DOE OSTI2024

Performance Characterization and Provenance of Distributed Task-based Workflows on HPC Platforms

Understanding performance and provenance of task-based workflows poses significant challenges, particularly in distributed configurations where resources are shared by multiple applications. Task-based workflow management systems further complicate performance predictability because of their dynamicity that subtly alters task execution order from run to run. In this paper we propose a layered characterization framework for performance and task provenance for Dask.distributed workflows running on high-performance computing (HPC) platforms. It collects data from jobs, the workflow management system, and the operating system to aid in understanding the performance of these workflows. Our approach encompasses three main contributions: first, an extension of Dask.distributed to capture high-fidelity task provenance using Mochi data services; second, the adaptation of the established HPC I/O characterization tool Darshan to gather high-fidelity I/O data, thereby enhancing the granularity of our analysis; and third, a framework to combine and process the collected data and provide helpful insights into performance characterization and reproducibility, alongside our lessons learned.

Dask↗
DOE OSTI2020

Dynamic Restoration Strategy for Distribution System Resilience Enhancement

In electric power distribution systems, distributed energy resources (DERs) can act as controllable power sources and support utility operators to minimize power outages after extreme weather events (e.g., hurricane, earthquake, wildfire) and thus help enhance the grid's resilience. Meanwhile, the influences of extreme events and the capabilities of DERs are dynamic and difficult to predict. Hence, the desired distribution system restoration strategy should be able to evolve according to real-time fault/dis-turbance information and the availability of DERs. In this paper, we propose a new dynamic restoration strategy for distribution systems to enhance system resilience against potential hazards. An efficient reconfiguration algorithm is developed to eliminate the use of integer variables to relieve the computational burden. Model predictive control is implemented to adjust the system topology and DER operation set points based on the updated fault information and DER forecasts. The effectiveness of the proposed restoration model in enhancing distribution system resilience is validated through an IEEE 123-bus test system. Simulation results also validate that the proposed restoration model can mitigate the occurrence of unexpected events and the fluctuations of DERs.

distributed energy resources (DERs)↗
DOE OSTI2023

Day-ahead continuous double auction-based peer-to-peer energy trading platform incorporating trading losses and network utilisation fee

Integration of distributed energy resources, such as photovoltaic solar (PV), introduces new opportunities to establish local energy market frameworks to improve renewable energy utilisation in residential sectors. Such peer-to-peer (P2P) energy trading refers to a local market structure where customers (and prosumers) interact to share excess PV generation to enhance the individual and community social welfare. In this work, a day-ahead continuous double auction (CDA)-based P2P market structure considering network losses and network utilisation fees was designed. Day-ahead PV energy is modelled using fractional integral polynomials and the output is forecasted using an autoregressive integrated moving average model for each market interval. Based on the customer load and excess PV energy, the CDA market is cleared using a bid/ask matching mechanism. The performance of the P2P market was evaluated by computing different welfare metrics while analysing the effect of network constraints. The results show that the designed CDA-based P2P market structure increases the social welfare of all participants by an average of 17.75% compared to the baseline for the presented cases. Moreover, the impact of the forecasting error between the day-ahead and real-time market was also quantified.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Framework for Large-scale Implementation of Wholesale-Retail Transactive Control Mechanism

Transactive energy is a control technique that uses market mechanisms to achieve desired control objectives. Several simulation studies and field demonstrations were carried out in recent years, but all focused on small scale systems and purpose-built simplified models which are not always capable of pointing out all the advantages and shortcomings of transac- tive energy methods. This work describes a co-simulation framework built on a hierarchical control architecture that allows for conducting studies of the impacts of a very large-scale deployment of transactive energy. The hierarchical transactive control architecture adopted in this work helps with alleviating computation and communication burden to facilitate a more effective large scale real-time market operation among device level resources and the system level operators. The co-simulation framework is evaluated an integrated power sys- tem model of unprecedented scale composed of the Western Electricity Coordination Council (WECC) transmission system with tens of thousands of distribution systems deployed with flexible device-level distributed energy resources (DERs) using off-the-shelf simulators.

Transactive energy, market-based controls, DER int↗
DOE OSTI2026

High-energy synchrotron X-ray multimodal computed tomography: enabling multiscale materials characterization at NSLS-II

We report the commissioning of a multimodal computed tomography experimental setup at the 28-ID-2 (XPD) beamline of the National Synchrotron Light Source II. This high-energy (>60 keV) resource features a tunable X-ray beam size ranging from several millimetres to a few micrometres and enables comprehensive characterization of high-Z materials—an essential capability for nuclear and advanced materials research. It provides four complementary computed tomography modalities: X-ray absorption, X-ray fluorescence, X-ray diffraction, and pair distribution function tomography. A case study using a custom-made heterogeneous sample demonstrates these abilities to simultaneously capture atomic, elemental, and morphological information. This unique combination of imaging, structural, and chemical sensitive methods provides a holistic approach to study complex materials with amorphous and crystalline systems across multiple length scales.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Federated Learning for Efficient Condition Monitoring and Anomaly Detection in Industrial Cyber-Physical Systems

Detecting and localizing anomalies in cyber-physical systems (CPS) has become increasingly challenging as systems grow in complexity, particularly due to varying sensor reliability and node failures in distributed environments. While federated learning (FL) offers a foundation for distributed model training, existing approaches lack mechanisms to handle these CPS-specific challenges. This paper presents an enhanced FL framework that introduces three key innovations: adaptive model aggregation based on sensor reliability, dynamic node selection for resource optimization, and Weibull-based checkpointing for fault tolerance. Our framework enables reliable condition monitoring while addressing the computational and reliability challenges of industrial CPS deployments. Experiments on NASA Bearing and Hydraulic System Datasets demonstrate superior performance over state-of-the-art FL methods, achieving 99.5% AUC-ROC in anomaly detection and maintaining accuracy under node failures. Statistical validation using Mann-Whitney (U) test confirms significant improvements (p < 0.05) in both detection accuracy and computational efficiency across diverse operational scenarios.1

Marfo, William [University of Texas at El Paso,Dep↗
DOE OSTI2021

Penetration Through Slots in Overmoded Cavities

A resonant cavity undergoes three distinct behaviors with increasing frequency: 1) fundamental modes, localized in frequency with well defined modal distribution; 2) undermoded region, where modes are still separated, but are sufficiently perturbed by small imperfections that their spectral positions (and distributions) are statistical in nature; and 3) overmoded region, where modes overlap, field distributions follow stochastic distributions, and the slot acts as if in free space. Understanding the penetration through slots in the overmoded region is of great interest, and is the focus of this article. Since full-wave solvers may not be able to provide a timely answer for very high frequencies due to a lack of memory and/or computation resources, we develop bounding methods to estimate worst-case average and maximum fields within the cavity. Finally, after discussing the bounding formulation, we compare its results to full-wave simulations at the first, second, and third resonance supported by the slot in the case of a cylindrical cavity. Note that the bounding formulation indicates that results are nearly independent of cavity shape: only the cavity volume, frequency, and cavity quality factor affect the overmoded region, making this formulation a powerful tool to assess electromagnetic interference and electromagnetic compatibility effects within cavities.

overmoded cavity↗
DOE OSTI2022

MIRACL Co-Simulation Platform Lab assets and tools integration

Pacific Northwest National Laboratory's (PNNL) co-simulation platform (CSP) for the Microgrids, Infrastructure Resilience, and Advanced Controls Launchpad (MIRACL) project, also known as MIRACL-CSP, is a functional layer designed and developed to oversee the operational exchanges of data at the application level to and from different resources residing on the MIRACL Data Hub shared platform. MIRACL-CSP allows virtual interactions between various data hub resources during co-simulation runtime.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Multiscale Electricity Modeling for Evaluating Carbon Capture and Sequestration Technologies (MEME-CCS)

This effort employs and adapts an existing, rigorous multiscale electricity modeling platform at the National Renewable Energy Laboratory (NREL) to evaluate carbon capture and sequestration (CCS) and negative emissions technologies (NET) from the ARPA-E FLECCS program. NREL's modeling platform includes the Regional Energy Deployment System (ReEDS) electric sector capacity expansion model, which projects future electricity generation mixes at sub-state-level resolution that are downscaled to the unit-level to enable hourly, zonal or nodal electricity production cost modeling in the PLEXOS model. The resulting hourly price data from PLEXOS is provided to technology developers under the ARPA-E FLECCS program to enable technology-specific economic analysis. The ReEDS-PLEXOS modeling suite is well-established for examining electric sector futures with high renewable energy penetrations, energy storage, electrification, and distributed generation. The key advancement proposed herein utilizes collaboration with CCS experts at the University of Wyoming and the FLECCS teams to create innovative methods for representing CCS and NET in the ReEDS and PLEXOS models. Expanded technology options, new operational parameterizations, and detailed data defining CO2 capture, transportation, and storage systems are being integrated into these models to allow an unprecedented combination of scope and resolution for exploring the future of CCS and NET. The resulting capabilities will take advantage of high-performance computing resources to permit wide-ranging scenario analysis to assess CCS and NET deployment under alternative CO2 prices, fossil fuel prices, electricity demand growth, and other electric sector characteristics. Final outcomes will include publicly available hourly grid operation and price data for any U.S. region of interest along with open-access capacity expansion tools for evaluating CCS/NET systems. These products will help an emerging CCS/NET industry in the United States by providing economics-driven guidance to technology developers while informing policy and investment decisions in the public and private sectors.

air capture↗