Design of False Data Injection Attack on Distributed Process Estimation
Not provided.
Computer Science↗
SEARCH · Engineering Papers
Results for “Distributed Processing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
SparseX - A tool for distributed processing of multidimensional sparse data
Developed for atmospheric science: specifically, a tool for processing sparse photon propagation data
Corzine, Riley Zhou↗
Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time
The Vera C. Rubin Observatory is preparing to execute the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time (LSST). Currently the final phase of construction is under way in the Chilean Andes, with the Observatory’s ten-year science mission scheduled to begin in 2025. Rubin’s 8.4-meter telescope will nightly scan the southern hemisphere collecting imagery in the wavelength range 320–1050 nm covering the entire observable sky every 4 nights using a 3.2 gigapixel camera, the largest imaging device ever built for astronomy. Automated detection and classification of celestial objects will be performed by sophisticated algorithms on high-resolution images to progressively produce an astronomical catalog eventually composed of 20 billion galaxies and 17 billion stars and their associated physical properties. In this article we present an overview of the system currently being constructed to perform data distribution as well as the annual campaigns which reprocess the entire image dataset collected since the beginning of the survey. These processing campaigns will utilize computing and storage resources provided by three Rubin data facilities (one in the US and two in Europe). Each year a Data Release will be produced and disseminated to science collaborations for use in studies comprising four main science pillars: probing dark matter and dark energy, taking inventory of solar system objects, exploring the transient optical sky and mapping the Milky Way. Also presented is the method by which we leverage some of the common tools and best practices used for management of large-scale distributed data processing projects in the high energy physics and astronomy communities. We also demonstrate how these tools and practices are utilized within the Rubin project in order to overcome the specific challenges faced by the Observatory.
79 ASTRONOMY AND ASTROPHYSICS↗
Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing
In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.
Hou, Shiyue↗
Techniques for collective operations in distributed systems
Various embodiments are generally directed to techniques for collective operations among compute nodes in a distributed processing set, such as by utilizing ring sets and local sets of the distributed processing set. In some embodiments, a ring set may include a subset of the distributed processing set in which each compute node is connected to a network with a separate router. In various embodiments, a local set may include a subset of the distributed processing set in which each compute node is connected to a network with a common router. In one or more embodiments, each compute node in a distributed processing set may belong to one ring set and one local set.
Langer, Akhil↗
Multi-Fidelity Gaussian Process for Distribution System Voltage Probabilistic Estimation with PVs
The increasing penetration of behind-the-meter PVs causes challenges to maintain voltage security due to the lack of distribution system visibility. This paper proposes a nonlinear autoregressive Gaussian process (NARGP) approach to fuse limited number of SCADA/AMI data together with historical pseudo measurements for distribution node voltage probabilistic estimation. The high-fidelity SCADA data are fused with the low-fidelity AMI and pseudo measurements by the autoregressive algorithm embedded in the Gaussian process. This allows us to use multi-fidelity data to achieve entire distribution system voltage visibility. Numerical results carried out on the IEEE 123node system demonstrate that the NARGP method is able to obtain high accuracy in estimating bus voltage and quantifying estimation uncertainties as compared to other approaches.
distribution system estimation↗
Evaluation of distributed process-based hydrologic model performance using only a priori information to define model inputs
Fully distributed, integrated surface–subsurface hydrological models (ISSHMs) have seen renewed interest due to availability of better software, high performance computing facilities, and high-resolution, spatially extensive data products. ISSHMs are valuable as tools for advancing system understanding as they can resolve multiple processes defined on the plot scale including three-dimensional interaction of surface water and groundwater. Here, we evaluated the performance of an ISSHM, the Advanced Terrestrial Simulator (ATS), on seven diverse catchments across the continental US using widely available data products to define model inputs without calibration. We compare the ATS-simulated streamflow and evapotranspiration with gauge observations and MODIS-derived evapotranspiration, respectively. Using the Kling-Gupta Efficiency (KGE) as metric, ATS with default data products performed reasonably well at 6 of 7 catchments for streamflow. However, in one of those 6 catchments ATS had poor performance on baseflow and ATS’s overall performance was thus judged to be inadequate despite the acceptable KGE. ATS performance for evapotranspiration was good in all 7 catchments using default data products. In the two catchments where ATS streamflow performance using default data products was not acceptable, the performance was significantly improved by using local information on subsurface properties below the soil. We also compare the model-simulated streamflow and evapotranspiration with the Sacramento soil moisture accounting (SAC-SMA) model, a semi-distributed model that was calibrated on a catchment-by-catchment basis. Uncalibrated ATS performance is comparable to the calibrated SAC-SMA model in terms of streamflow while ATS performance is similar to or better (much better in certain catchments) in reproducing MODIS-derived evapotranspiration. Reasonably good performance of ATS without catchment-specific calibration provides new confidence in the ISSHM class of models and community data products as tools for advancing understanding of watershed function in a changing environment.
54 ENVIRONMENTAL SCIENCES↗
Dynamic frequency control in power networks
Node controllers in power distribution networks in accordance with embodiments of the invention enable dynamic frequency control. One embodiment includes a node controller comprising a network interface a processor; and a memory containing a frequency control application; and a plurality of node operating parameters describing the operating parameters of a node, where the node is selected from a group consisting of at least one generator node in a power distribution network wherein the processor is configured by the frequency control application to calculate a plurality of updated node operating parameters using a distributed process to determine the updated node operating parameter using the node operating parameters, where the distributed process controls network frequency in the power distribution network; and adjust the node operating parameters.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Enhanced Signal Processing of Distributed Brillouin Fiber Sensors using a Decoupled Radial Basis Function Network
A novel decoupled radial basis function network (D-RBFN) is proposed to accelerate signal processing and address the big data challenges associated with ultra-long distance Brillouin optical time-domain analysis (BOTDA) systems. The proposed frame- work is demonstrated on a dataset measured over a 100 km distance using a bi-directional Raman assisted BOTDA system.
Venketeswaran, Abhishek↗
Identifying electrochemical processes by distribution of relaxation times in proton exchange membrane electrolyzers
Distribution of relaxation time (DRT) is used to interpret electrochemical impedance spectroscopy (EIS) for proton exchange membrane (PEM) water electrolyzers, with an attempt to separate overlapped relaxation processes in Nyquist plots. By varying operating conditions and catalyst loadings, four main relaxation peaks arising from EIS can be identified and successfully separated from low to high frequencies as (P1) mass transport, (P2) oxygen evolution reaction kinetics, (P3) reaction kinetics (with faster time constant than P2), and (P4) ionic transport. Here, the shape, height, and frequency of the DRT peaks change with different membrane electrode assembly (MEA) configurations. Electron microscopy reveals distinct features from the cross-sectioned MEAs which verify critical DRT results in that increasing the iridium (Ir)-anode loading from 0.2 mgIr/cm 2 to 1.5 mgIr/cm 2 reduces kinetic losses due to higher site-access; a thick and compacted anode, however, also triggers higher ohmic resistances from membrane/catalyst layer hydration and increases transport losses due to longer ionomer pathways. DRT provides higher resolution to EIS for deconvoluting processes with different relaxation times and the quantification of DRT peaks improves the accounting of total losses from each process.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Hardening DOE R&D Software Tools for Web-based Visualization SBIR Phase I Final Report
Ubiquitous web-based visualization is essential to delivering large-scale data visualization to various stakeholders, from the scientist to the board member. These stakeholders will not tolerate a stalled application or a pop-up window asking them to wait for the processing to complete. They require a responsive and interactive visualization environment with high-quality imagery suitable for detailed analysis and boardroom presentations. At Kitware, Inc., we have accomplished web visualization to this point, leveraging state-of-the-art tools like HTML5, CSS3, SVG, Canvas, and WebGL. Solutions that leverage a combination of these technologies are necessary to handle workloads that vary significantly in data size efficiently. However, it is not always practical to move large data to the web client for visualization. Kitware's ParaView as a Service combines client-side visualization using both distributed processing and remote rendering on big data impractical to move. Existing distributed processing and remote rendering solution's interactivity is below the expectations of web-based applications. Our project examined proposed solutions to the areas outlined above in ParaView as a Service. We have investigated concurrent pipelines, streaming images, progressive rendering, and optimization of algorithms and data movement to address these concerns. For the Phase I project, we completed the proposed work plan. As a result, the project produced three prototypes of essential importance for web visualization and the ParaView as a Service community. We created a simple desktop application for an interactive streamline placement prototype, a web-based interactive streamline placement prototype, and a web-based progressive rendering utilizing raytracing prototype. These prototypes relied on the hardening of emerging software toolkits funded by the Department of Energy (DOE) Advanced Scientific Computing Research (ASCR) program (such as ParaView, VTK-m, and Mochi). We blended these components into web-based visualization prototypes that meet the industry's expectations for interactivity and responsiveness. The Phase I project had four essential focus areas: 1. Develop prototype ParaView as a Service backend server using asynchronous, non-blocking design principles. 2. Develop a prototype web application that uses the ParaView as a Service backend server for remote data visualization. 3. Implement image streaming with encoding/compression and progressive rendering capabilities in the proposed platform. 4. Evaluate the prototype developed and summarize observations, including the challenges and pitfalls of our approach. After our successful completion of Phase I, we are strongly positioned to propose a successful Phase II project.
Geveci, Berk↗
Decoding Resilience by Modeling Outage and Restoration Processes in Distribution Grids: A Pittsburgh Case Study
Climate-induced extreme weather events, such as floods and heatwaves, pose significant challenges to the resilience of urban power distribution grids. This paper examines the outage and restoration dynamics of Pittsburgh's power grid during the flood event in April 2024 and three heatwaves in June, July, and August 2024. We introduce a comprehensive modeling framework that integrates outage and restoration processes, enabling the quantification of resilience metrics, including total customer-hours of power outage, maximum residual values, and restoration durations across various ZIP codes. Our analysis highlights spatial disparities in outage impacts and restoration efficiencies, with ZIP Code 15222 experiencing the highest cumulative disruptions, while ZIP Code 15217 shows lower susceptibility. By linking statistical trends with weather events, this study underscores the critical need for targeted infrastructure upgrades and advanced restoration strategies. Furthermore, the proposed framework offers valuable insights for planning and managing resilient power systems in the face of increasing climate stress.
24 POWER TRANSMISSION AND DISTRIBUTION↗
The Role of SnO2 Processing on Ionic Distribution in Double-Cation-Double Halide Perovskites
Moving toward a future of efficient, accessible, and less carbon-reliant energy devices has been at the forefront of energy research innovations for the past 30 years. Metal-halide perovskite (MHP) thin films have gained significant attention due to their flexibility of device applications and tunable capabilities for improving power conversion efficiency. Serving as a gateway to optimize device performance, consideration must be given to chemical synthesis processing techniques. Therefore, how does common substrate processing techniques influence the behavior of MHP phenomena such as ion migration and strain? Here, we demonstrate how a hybrid approach of chemical bath deposition (CBD) and nanoparticle SnO2 substrate processing significantly improves the performance of (FAPbI3)0.97(MAPbBr3)0.03 by reducing micro-strain in the SnO2 lattice, allowing distribution of K+ from K-Cl treatment of substrates to passivate defects formed at the interface and produce higher current in light and dark environments. X-ray diffraction reveals differences in lattice strain behavior with respect to SnO2 substrate processing methods. Through use of conductive atomic force microscopy (c-AFM), conductivity is measured spatially with MHP morphology, showing higher generation of current in both light and dark conditions for films with hybrid processing. Additionally, time-of-flight secondary ionization mass spectrometry (ToF-SIMS) observed the distribution of K+ at the perovskite/SnO2 interface, indicating K+ passivation of defects to improve the power conversion efficiency (PCE) and device stability. We show how understanding the role of ion distribution at the SnO2 and perovskite interface can help reduce the creating of defects and promote a more efficient MHP device.
conductive atomic force microscopy↗
Real-Time Asynchronous Information Processing in Distributed Power Systems Control
Not provided.
Engineering↗
PhytoOracle: Scalable, modular phenomics data processing pipelines
As phenomics data volume and dimensionality increase due to advancements in sensor technology, there is an urgent need to develop and implement scalable data processing pipelines. Current phenomics data processing pipelines lack modularity, extensibility, and processing distribution across sensor modalities and phenotyping platforms. To address these challenges, we developed PhytoOracle (PO), a suite of modular, scalable pipelines for processing large volumes of field phenomics RGB, thermal, PSII chlorophyll fluorescence 2D images, and 3D point clouds. PhytoOracle aims to ( i ) improve data processing efficiency; ( ii ) provide an extensible, reproducible computing framework; and ( iii ) enable data fusion of multi-modal phenomics data. PhytoOracle integrates open-source distributed computing frameworks for parallel processing on high-performance computing, cloud, and local computing environments. Each pipeline component is available as a standalone container, providing transferability, extensibility, and reproducibility. The PO pipeline extracts and associates individual plant traits across sensor modalities and collection time points, representing a unique multi-system approach to addressing the genotype-phenotype gap. To date, PO supports lettuce and sorghum phenotypic trait extraction, with a goal of widening the range of supported species in the future. At the maximum number of cores tested in this study (1,024 cores), PO processing times were: 235 minutes for 9,270 RGB images (140.7 GB), 235 minutes for 9,270 thermal images (5.4 GB), and 13 minutes for 39,678 PSII images (86.2 GB). These processing times represent end-to-end processing, from raw data to fully processed numerical phenotypic trait data. Repeatability values of 0.39-0.95 (bounding area), 0.81-0.95 (axis-aligned bounding volume), 0.79-0.94 (oriented bounding volume), 0.83-0.95 (plant height), and 0.81-0.95 (number of points) were observed in Field Scanalyzer data. We also show the ability of PO to process drone data with a repeatability of 0.55-0.95 (bounding area).
59 BASIC BIOLOGICAL SCIENCES↗
Microchannel Reactor for Ethanol to Butene: CRADA 503 [Abstract only]
A key challenge facing most bioprocessing operations is that multiple unit operations are required, thereby resulting in complex, energy-intensive, and expensive processes. Further, biomass transportation costs drive the need for smaller, distributed processing plants. To incorporate the smaller scales desirable for biomass, novel processes must be developed with reduced capital costs. With over 20 years of experience in the development and commercialization of microchannel reactor technology, Oregon State University will partner with Pacific Northwest National Laboratory to demonstrate a microchannel reactor with lower capital costs for an alcohol-to-jet (ATJ) process technology that is currently being commercialized by LanzaTech. Ethanol can be produced from biomass feedstocks such as LanzaTech’s proprietary biochemical process using carbon from a number of possible feedstocks; syngas generated from biomass resources (e.g., MSW, organic industrial waste, agriculture waste) or reformed biogas, or from other biomass feedstocks such as corn kernel fiber. Ethanol then undergoes catalytic dehydration to form ethylene followed by a two-step oligomerization, hydrogenation, and fractionation to control the hydrocarbon product slate to the jet-range. Successful process development aided by a market pull for low carbon aviation fuel has spurred scale-up and commercial demonstration. However, Sustainable Aviation Fuel is a very price sensitive market and improved economics through process intensification will make the current ATJ process even more attractive. Recent efforts at PNNL have culminated in the development of a new catalyst technology for the conversion of ethanol to n-butene-rich olefins. A greater than 90% conversion, total olefin selectivity of 80-90% (n-butene selectivity ~60%), and good stability over a 100 hour test duration has been demonstrated at the bench scale. Producing butene-rich olefins directly from ethanol with high yield is new and impactful because the higher olefins can be selectively oligomerized to distillate-range hydrocarbons, thus eliminating one process step from the current ATJ process. Further, coupling the severely endothermic ethanol dehydration with exothermic C-C bond formation results in more energy efficient processing. Additional intensification and energy savings will stem from incorporating this new ethanol to n-butene catalyst technology within the ATJ process implemented using a microchannel reactor platform. Due to recent advances in microchannel manufacturing methods and associated cost reductions we believe the time is right to adapt this technology toward new commercial bioconversion applications.
02 PETROLEUM↗
The importance of explicitly representing the streambed in watershed models
Abstract The streambed is the critical interface between the aquatic and terrestrial systems and hosts important biogeochemical hot spots within river corridors. Although the streambed characteristics are significantly different from those of its surrounding soil, the streambed itself has not been explicitly represented in watershed models. Here, we explicitly incorporated a streambed layer into an integrated hydrologic model through model parameterization and discretization. We examined the hydrological effects of streambed characteristics, including hydraulic conductivity ( K ), layer thickness, and resolution, on the exchange fluxes across the streambed as well as the streamflow at the watershed outlet. The numerical experiments were performed in the American River Watershed, a headwater, mountainous watershed within the Yakima River Basin in central Washington. Despite having a negligible effect on the watershed streamflow, an explicit representation of the streambed with distinctive properties dramatically changed the magnitude and variability of the exchange flux. In general, a larger streambed K along with a thicker streambed layer induced larger exchange fluxes. The exchange flux was most sensitive to the streambed resolution. A finer streambed resolution increased exchange fluxes per unit area while reducing the overall exchange volumes across the entire streambed. The amount of baseflow decreased by 6% as the streambed resolution increased from 250 to 50 m. This finding is important because these hydrological changes may, in turn, affect the exchange of nutrients and contaminants between surface water and groundwater and the associated biogeochemical processes. Our work demonstrated the importance of representing streambeds in fully distributed, process‐based watershed models to better capture the exchange flow dynamics in river corridors.
54 ENVIRONMENTAL SCIENCES↗
Simulation and Optimization of Volatile Fatty Acid Upgrading Strategies for Sustainable Transportation Fuel Production
Underutilized wet waste is a promising feedstock for production of carbon-neutral or carbon-negative liquid transportation fuels. Arrested methanogenesis of wet wastes by microbes, which produces mixtures of C2-C8 carboxylic acids (volatile fatty acids, VFAs), is practiced industrially at a pilot scale and the VFA products can be catalytically upgraded to molecules suitable for use as transportation fuels (alcohols and alkanes) via sequential ketonization and hydrogenation, both steps of which have been demonstrated with high (>90%) yield at the lab scale. We present in this work a simulation-based decision-making framework which evaluates upgrading strategies of VFAs to alcohol- and hydrocarbon-based neat and blended diesel, jet, and gasoline fuels. The processes utilize sequential upgrading steps (ketonization and hydrogenation) accompanied by one distillation step (either separating VFAs before ketonization or alcohols after ketone hydrogenation) to form two liquid streams: a light alcohol stream best suited as a light-duty fuel and a heavy alcohol or hydrocarbon stream best suited as a diesel or jet fuel. Suitability of the liquid products as transportation fuels and maximum blending levels with petrofuels comes from simulation of critical fuel properties (boiling point, flash point, lower heating value, viscosity, melting point, water solubility, and cetane/octane number). Catalytic upgrading steps and fuel property mixing models used by the simulation are experimentally validated. We demonstrate the flexibility of the decision-making algorithm by evaluating processing scenarios for experimentally-observed VFA profiles with varied carbon chain length distributions. Processing scenarios are optimized for either maximum total renewable carbon utilization or production of heavy-duty (jet or diesel) fuels. We evaluate the economic viability and CO2 emissions of each proposed upgrading scenario using techno-economic and life cycle analyses. The decision-making framework developed in this work can also be used to down-select promising strategies for upgrading of other bio-based feedstocks to sustainable fuels and chemicals.
aviation fuel↗