Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Performance Analysis of Data Processing in Distributed File Systems with Near Data Processing

In the era of big data, the escalating volume and velocity of data generation pose significant challenges in data processing. Traditional systems like Spark and Hadoop manage the increasing amount and velocity of data by improving data placement and processing speeds. However, they face inherent limitations due to the essential data movement required for processing. In this paper, we explore the Skyhook framework, a novel extension of the Ceph distributed system, which significantly reduces the need for data movement. We present an extensive case study using the Skyhook framework, applying it with the TPC-H and K-means clustering algorithms. More specifically, we leverage the TPC-H benchmark to distinguish between CPU-intensive and I/O-intensive tasks. We explore the integration of K-means clustering into SQL, coupled with a near-data processing system to offload the computational burden of the K-means clustering algorithm to storage nodes. We conduct a comprehensive performance evaluation of distributed data processing applications across three processing approaches: traditional layout (baseline), optimized layout, and near-data processing. Additionally, we introduce the use of the FIO tool to simulate real-world system workloads, enabling the measurement of performance metrics such as average latency and CPU utilization. Our research is a significant advance in understanding how to optimize data processing systems to meet the demands of the modern data landscape.

Hou, Shiyue↗
DOE OSTI2021

Enhancement of Distribution System State Estimation Using Pruned Physics-Aware Neural Networks: Preprint

Realizing complete observability in the three-phase distribution system remains a challenge that hinders the implementation of classical state estimation algorithms. In this paper, a new method so-called pruned physics-aware neural network (P2N2) is developed to improve the voltage estimation accuracy in the distribution system. The method relies on the physical grid topology, which is used to design the connections between different hidden layers of a neural network model. To verify the proposed method, a numerical simulation based on one-year smart meter data of load consumptions for threephase power flow is developed to generate the measurement and voltage state data. The IEEE 123 node system is selected as the test network to benchmark the proposed algorithm against the classical weighted least squares (WLS). Numerical results show that P2N2 outperforms WLS, in terms of data redundancy and estimation accuracy.

distribution systems state estimation↗
NASA NTRS2000

A New Fast Algorithm to Completely Account for Non-Lambertian Surface Reflection of The Earth

Surface bidirectional reflectance distribution function (BRDF) influences not only radiance just about the surface, but that emerging from the top of the atmosphere (TOA). In this study we propose a new, fast and accurate, algorithm CASBIR (correction for anisotropic surface bidirectional reflection) to account for such influences on radiance measured above TOA. This new algorithm is based on a 4-stream theory that separates the radiation field into direct and diffuse components in both upwelling and downwelling directions. This is important because the direct component accounts for a substantial portion of incident radiation under a clear sky, and the BRDF effect is strongest in the reflection of the direct radiation reaching the surface. The model is validated by comparison with a full-scale, vector radiation transfer model for the atmosphere-surface system. The result demonstrates that CASBIR performs very well (with overall relative difference of less than one percent) for all solar and viewing zenith and azimuth angles considered in wavelengths from ultraviolet to near-infrared over three typical, but very different surface types. Application of this algorithm includes both accounting for non-Lambertian surface scattering on the emergent radiation above TOA and a potential approach for surface BRDF retrieval from satellite measured radiance.

Qin, Wen-Han↗
NASA NTRS1989

Online submicron particle sizing by dynamic light scattering using autodilution

Efficient production of a wide range of commercial products based on submicron colloidal dispersions would benefit from instrumentation for online particle sizing, permitting real time monitoring and control of the particle size distribution. Recent advances in the technology of dynamic light scattering (DLS), especially improvements in algorithms for inversion of the intensity autocorrelation function, have made it ideally suited to the measurement of simple particle size distributions in the difficult submicron region. Crucial to the success of an online DSL based instrument is a simple mechanism for automatically sampling and diluting the starting concentrated sample suspension, yielding a final concentration which is optimal for the light scattering measurement. A proprietary method and apparatus was developed for performing this function, designed to be used with a DLS based particle sizing instrument. A PC/AT computer is used as a smart controller for the valves in the sampler diluter, as well as an input-output communicator, video display and data storage device. Quantitative results are presented for a latex suspension and an oil-in-water emulsion.

Nicoli, David F.↗
NASA NTRS2022

Stability of the Dual-Frequency Radar Equations and a New Method Applied to the GPM’s Dual-Frequency Precipitation Radar (DPR) Data

A new algorithm is proposed that estimates two parameters of the particle size distribution (PSD) at each range bin from the Global Precipitation Measurement’s (GPM’s) Dual-frequency Precipitation Radar data. The equation that expresses the relationship of the PSD parameters between adjacent range bins is derived. By including the attenuation effect within the bin in the discretized equation, the new algorithm alleviates the double-solution problem when attenuation within the bin is sufficiently large. The stability of the solutions to the equation depends on the value of the mean diameter D(sub m) and its gradient with respect to range in the case of liquid precipitation. If the critical diameter above which the dual-frequency ratio (DFR) of radar reflectivity factors becomes a monotonically increasing function of D(sub m), the backward processing of the equation provides a stable or moderately diverging solution, unlike the forward processing that often gives unstable solutions. To provide a set of initial conditions without using the surface reference technique (SRT) in the backward processing, an initialization method using the Hitschfeld-Bordan attenuation correction method is proposed and tested. The proposed algorithm may provide a tool for investigating the assumptions used in various algorithms.

GPM↗
NASA NTRS2006

Improved Soundings and Error Estimates using AIRS/AMSU Data

AIRS was launched on EOS Aqua on May 4, 2002, together with AMSU A and HSB, to form a next generation polar orbiting infrared and microwave atmospheric sounding system. The primary products of AIRS/AMSU are twice daily global fields of atmospheric temperature-humidity profiles, ozone profiles, sea/land surface skin temperature, and cloud related parameters including OLR. The sounding goals of AIRS are to produce 1 km tropospheric layer mean temperatures with an rms error of 1 K, and layer precipitable water with an rms error of 20 percent, in cases with up to 80 percent effective cloud cover. The basic theory used to analyze AIRS/AMSU/HSB data in the presence of clouds, called the at-launch algorithm, and a post-launch algorithm which differed only in the minor details from the at-launch algorithm, have been described previously. The post-launch algorithm, referred to as AIRS Version 4.0, has been used by the Goddard DAAC to analyze and distribute AIRS retrieval products. In this paper we show progress made toward the AIRS Version 5.0 algorithm which will be used by the Goddard DAAC starting late in 2006. A new methodology has been developed to provide accurate case by case error estimates for retrieved geophysical parameters and for the channel by channel cloud cleared radiances used to derive the geophysical parameters from the AIRS/AMSU observations. These error estimates are in turn used for quality control of the derived geophysical parameters and clear column radiances. Improvements made to the retrieval algorithm since Version 4.0 are described as well as results comparing Version 5.0 retrieval accuracy and spatial coverage with those obtained using Version 4.0.

Susskind, Joel↗
NASA NTRS2004

Guaranteed spatial initialization of distributed spacecraft formations

In a precious paper the authors developed a formation initialization (FI) algorithm for a deep space, N-spacecraft formation. It was demonstrated analytically that this FI contribution of this paper is to extend this planar guarantee to deep space formations with arbitrary initial conditions. As part of the guarantee of initialization, a bound on the time-to-initialize is obtained. The guaranteed FI algorithm is then demonstrated for a two-spacecraft formation with realistic deep space mission constraints (e.g. limited field-of-view relative sensors and attitude constraints). The two-spacecraft scenario is challenging in that it has the least relative sensor field-of-view overlap. Finally, for this scenario, the distribution of time-to-initialize is characterized through a 150,000-case Monte Carlo analysis.

initialization↗
DOE OSTI2025

ED-cPSD: Fast Phase-Size Distribution via Sequential Erosion-Dilation

The Erosion-Dilation continuous Phase-Size Distribution, ED-cPSD, is an application for calculating continuous pore and particle-size distribution from digital reconstructions and/or image-based structural data. It is based on the erosion-dilation continuous phase-size distribution method. A continuous size distribution is a measure of the probability density of finding a particle or pore of a certain size. These distributions are of interest in any field of study involving porous media, including but not limited to electrochemistry, petroleum engineering, geology, and food science. The algorithm behind the software provides a computationally efficient way to calculate phase-size distributions for large domains. For a 3D battery electrode reconstruction with 1.3 x 10 8 voxels, the particle size distribution is derived in under 2 min on a desktop, while also retaining flexibility and computational efficiency for HPC-scale multi-threading. The software can handle structures with over 10 9 voxels. The algorithm is roughly 280 times faster than a previous version on the same task.

Characterization↗
DOE OSTI2020

Understanding performance variability in standard and pipelined parallel Krylov solvers

In this work, we collect data from runs of Krylov subspace methods and pipelined Krylov algorithms in an effort to understand and model the impact of machine noise and other sources of variability on performance. We find large variability of Krylov iterations between compute nodes for standard methods that is reduced in pipelined algorithms, directly supporting conjecture, as well as large variation between statistical distributions of runtimes across iterations. Based on these results, we improve upon a previously introduced nondeterministic performance model by allowing iterations to fluctuate over time. We present our data from runs of various Krylov algorithms across multiple platforms as well as our updated non-stationary model that provides good agreement with observations. We also suggest how it can be used as a predictive tool.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2015

The Suomi National Polar-Orbiting Partnership (SNPP): Continuing NASA Research and Applications

The Suomi National Polar-orbiting Partnership (SNPP) satellite was successfully launched into a polar orbit on October 28, 2011 carrying 5 remote sensing instruments designed to provide data to improve weather forecasts and to increase understanding of long-term climate change. SNPP provides operational continuity of satellite-based observations for NOAA's Polar-orbiting Operational Environmental Satellites (POES) and continues the long-term record of climate quality observations established by NASA's Earth Observing System (EOS) satellites. In the 2003 to 2011 pre-launch timeframe, NASA's SNPP Science Team assessed the adequacy of the operational Raw Data Records (RDRs), Sensor Data Records (SDRs), and Environmental Data Records (EDRs) from the SNPP instruments for use in NASA Earth Science research, examined the operational algorithms used to produce those data records, and proposed a path forward for the production of climate quality products from SNPP. In order to perform these tasks, a distributed data system, the NASA Science Data Segment (SDS), ingested RDRs, SDRs, and EDRs from the NOAA Archive and Distribution and Interface Data Processing Segments, ADS and IDPS, respectively. The SDS also obtained operational algorithms for evaluation purposes from the NOAA Government Resource for Algorithm Verification, Independent Testing and Evaluation (GRAVITE). Within the NASA SDS, five Product Evaluation and Test Elements (PEATEs) received, ingested, and stored data and performed NASA's data processing, evaluation, and analysis activities. The distributed nature of this data distribution system was established by physically housing each PEATE within one of five Climate Analysis Research Systems (CARS) located at either at a NASA or a university institution. The CARS were organized around 5 key EDRs directly in support of the following NASA Earth Science focus areas: atmospheric sounding, ocean, land, ozone, and atmospheric composition products. The PEATES provided the system level interface with members of the NASA SNPP Science Team and other science investigators within each CARS. A sixth Earth Radiation Budget CARS was established at NASA Langley Research Center (NASA LaRC) to support instrument performance, data evaluation, and analysis for the SNPP Clouds and the Earth's Radiant Budget Energy System (CERES) instrument. Following the 2011 launch of SNPP, spacecraft commissioning, and instrument activation, the NASA SNPP Science Team evaluated the operational RDRs, SDRs, and EDRs produced by the NOAA ADS and IDPS. A key part in that evaluation was the NASA Science Team's independent processing of operational RDRs and SDRs to EDRs using the latest NASA science algorithms. The NASA science evaluation was completed in the December 2012 to April 2014 timeframe with the release of a series of NASA Science Team Discipline Reports. In summary, these reports indicated that the RDRs produced by the SNPP instruments were of sufficiently high quality to be used to create data products suitable for NASA Earth System science and applications. However, the quality of the SDRs and EDRs were found to vary greatly when considering suitability for NASA science. The need for improvements in operational algorithms, adoption of different algorithmic approaches, greater monitoring of on-orbit instrument calibration, greater attention to data product validation, and data reprocessing were prominent findings in the reports. In response to these findings, NASA, in late 2013, directed the NASA SNPP Science Team to use SNPP instrument data to develop data products of sufficiently high quality to enable the continuation of EOS time series data records and to develop innovative, practical applications of SNPP data. This direction necessitated a transition of the SDS data system from its pre-launch assessment mode to one of full data processing and production. To do this, the PEATES, which served as NASA's data product testing environment during the prelaunch and early on-orbit periods, were transitioned to Science Investigator-led Processing Systems (SIPS). The distributed data architecture was maintained in this new system by locating the SIPS at the same institutions at which the CARS and PEATES were located. The SIPS acquire raw SNPP instrument Level 0 (i.e. RDR) data over the full SNPP mission from the NOAA ADS and IDPS through the NASA SDS Data Distribution and Depository Element (SD3E). The SIPS process those data into NASA Level 1, Level 2, and global, gridded Level 3 standard products using peer-reviewed algorithms provided by members of the NASA Science Team. The SIPS work with the NASA SNPP Science Team in obtaining enhanced, refined, or alternate real-time algorithms to support the capabilities of the Direct Readout Laboratory (DRL). All data products, algorithm source codes, coefficients, and auxiliary data used in product generation are archived in an assigned NASA Distributed Active Archive Center (DAAC).

remote sensing↗
DOE OSTI2023

Estimating and Calibrating DER Model Parameters Using Levenberg–Marquardt Algorithm in Renewable Rich Power Grid

The proliferation of inverter-based distributed energy resources (IBDERs) has increased the number of control variables and dynamic interactions, leading to new grid control challenges. For stability analysis and designing appropriate protection controls, it is important that IBDER models are accurate. This paper focuses on the accurate estimation and parameter calibration of DER_A, a recently proposed aggregated IBDER model. In particular, we focus on the parameters of the reactive power–voltage regulation module. We formulate the problem of parameter tuning as a non-linear least square minimization problem and solve it using the Levenberg–Marquardt (LM) method. The LM method is primarily chosen due to its flexibility in adaptively selecting between the steepest descent and Gauss–Newton methods through a damping parameter. The LM approach is used to minimize the error between the actual measurements and the estimated response of the model. Further, the computational challenges posed by the numerical calculation of the Jacobian are tackled using a quasi-Newton root-finding approach. The proposed method is validated on a real feeder model in the northeastern part of the United States. The feeder is modeled in OpenDSS and the measurements thus obtained are fed to the DER_A model for calibration. The simulation results indicate that our approach is able to successfully calibrate the relevant model parameters quickly and with high accuracy, with a total sum of square error of 3.57 × 10 –7 .

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS1986

On compactness of admissible parameter sets: Convergence and stability in inverse problems for distributed parameter systems

A series of numerical examples is reported and several algorithms compared for estimation of coefficients in differential equation models. Unconstrained, constrained and Tikhonov regularization methods are tested for their behavior with regard to both convergence (of approximation methods for the states and parameters) and stability (continuity of the estimates with respect to perturbations in the data or observed states).

Banks, H. T.↗
DOE OSTI2021

Proof-of-concept of a reinforcement learning framework for wind farm energy capture maximization in time-varying wind

Here, we present a proof-of-concept distributed reinforcement learning framework for wind farm energy capture maximization. The algorithm we propose uses Q-Learning in a wake-delayed wind farm environment and considers time-varying, though not yet fully turbulent, wind inflow conditions. These algorithm modifications are used to create the Gradient Approximation with Reinforcement Learning and Incremental Comparison (GARLIC) framework for optimizing wind farm energy capture in time-varying conditions, which is then compared to the FLOw Redirection and Induction in Steady State (FLORIS) static lookup table wind farm controller baseline.

17 WIND ENERGY↗
NASA NTRS2017

Feature Detection and Curve Fitting Using Fast Walsh Transforms for Shock Tracking: Applications

Walsh functions form an orthonormal basis set consisting of square waves. Square waves make the system well suited for detecting and representing functions with discontinuities. Given a uniform distribution of 2p cells on a one-dimensional element, it has been proven that the inner product of the Walsh Root function for group p with every polynomial of degree < or = (p - 1) across the element is identically zero. It has also been proven that the magnitude and location of a discontinuous jump, as represented by a Heaviside function, are explicitly identified by its Fast Walsh Transform (FWT) coefficients. These two proofs enable an algorithm that quickly provides a Weighted Least Squares fit to distributions across the element that include a discontinuity. The detection of a discontinuity enables analytic relations to locally describe its evolution and provide increased accuracy. Time accurate examples are provided for advection, Burgers equation, and Riemann problems (diaphragm burst) in closed tubes and de Laval nozzles. New algorithms to detect up to two C0 and/or C1 discontinuities within a single element are developed for application to the Riemann problem, in which a contact discontinuity and shock wave form after the diaphragm bursts.

Gnoffo, Peter A.↗
NASA NTRS1992

Experiments on active isolation using distributed PVDF error sensors

A control system based on a two-channel narrow-band LMS algorithm is used to isolate periodic vibration at low frequencies on a structure composed of a rigid top plate mounted on a flexible receiving plate. The control performance of distributed PVDF error sensors and accelerometer point sensors is compared. For both sensors, high levels of global reduction, up to 32 dB, have been obtained. It is found that, by driving the PVDF strip output voltage to zero, the controller may force the structure to vibrate so that the integration of the strain under the length of the PVDF strip is zero. This ability of the PVDF sensors to act as spatial filters is especially relevant in active control of sound radiation. It is concluded that the PVDF sensors are flexible, nonfragile, and inexpensive and can be used as strain sensors for active control applications of vibration isolation and sound radiation.

Lefebvre, S.↗
NASA NTRS1992

Application of Laser Imaging for Bio/geophysical Studies

SPARTA, Inc. has developed a low-cost, portable laser imager that, among other applications, can be used in bio/geophysical applications. In the application to be discussed here, the system was utilized as an imaging system for background features in a forested locale. The SPARTA mini-ladar system was used at the International Paper Northern Experimental Forest near Howland, Maine to assist in a project designed to study the thermal and radiometric phenomenology at forest edges. The imager was used to obtain data from three complex sites, a 'seed' orchard, a forest edge, and a building. The goal of the study was to demonstrate the usefulness of the laser imager as a tool to obtain geometric and internal structure data about complex 3-D objects in a natural background. The data from these images have been analyzed to obtain information about the distributions of the objects in a scene. A range detection algorithm has been used to identify individual objects in a laser image and an edge detection algorithm then applied to highlight the outlines of discrete objects. An example of an image processed in such a manner is shown. Described here are the results from the study. In addition, results are presented outlining how the laser imaging system could be used to obtain other important information about bio/geophysical systems, such as the distribution of woody material in forests.

Hummel, J. R.↗
DOE OSTI2022

Retrieval of the sea spray aerosol mode from submicron particle size distributions and supermicron scattering during LASIC

Abstract. Improved quantification of sea spray aerosol concentration and size is important for determining aerosol effects on clouds and the climate, though attempts to accurately capture the size distribution of the sea spray mode remain limited by the availability of supermicron size distributions. In this work, we introduce a new approach to retrieving lognormal mode fit parameters for a sea spray aerosol mode by combining submicron size distributions with supermicron scattering measurements using a Mie inversion. Submicron size distributions were measured by an ultra-high-sensitivity aerosol spectrometer (UHSAS), and supermicron scattering was taken as the difference between <10 µm and <1 µm three-wavelength integrating nephelometer measurements (NEPH). This UHSAS-NEPH method was applied during background marine periods of the Department of Energy Atmospheric Radiation Measurement Layered Atlantic Smoke Interactions with Clouds (LASIC) campaign on Ascension Island (November 2016–May 2017), when the contribution of sea spray aerosol was expected to represent a large fraction of the aerosol mass and total scattering. Lognormal sea spray modal parameters were retrieved from comparisons between nephelometer measurements and a lookup table of Mie theory-simulated scattering coefficients for low-error solutions that minimized the 0.4–1 µm residual in the UHSAS size distribution. We evaluated the UHSAS-NEPH method with a set of clean marine measurements in the North Atlantic that included supermicron size and chemical measurements, showing that measured supermicron size distributions are needed to constrain the sea spray number concentration but that mass concentration was reasonably characterized using supermicron scattering. For LASIC, the UHSAS-NEPH method retrieved sea spray mode properties for approximately 88 % of the background marine times when the scattering variability and total particle concentration were low (<± 5 Mm−1 and <400 cm−3, respectively), with mass mean diameter ranging from 0.6 to 1.9 µm (1.47 ± 0.17 µm), modal width ranging from 1.1 to 3.97 (2.4±0.3), and mass concentration ranging from 0.18 to 23.0 µg m−3 (8.37. ± 4.1 µg m−3). The measured nephelometer scattering at three wavelengths was found to constrain the mode width marginally at the largest particle sizes in the absence of additional size and chemical measurements for defining parameters for the Mie solutions. Comparing UHSAS-NEPH retrievals to those of a fitting algorithm applied only to the submicron UHSAS number size distribution showed that correlations between retrieved mass concentration and the available mass-based sea spray tracers (coarse scattering, wind speed, and chloride) are low when supermicron measurements are not considered. This work demonstrates the added value of supermicron scattering measurements for retrieving reasonable sea spray mass concentrations, providing the best-available observationally constrained estimate of the sea spray mode properties when supermicron size distribution measurements are not available.

54 ENVIRONMENTAL SCIENCES↗