Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

The Development of Two Science Investigator-led Processing Systems (SIPS) for NASA's Earth Observation System (EOS)

In 2001, NASA Goddard Space Flight Center's Laboratory for Terrestrial Physics started the construction of a science Investigator-led Processing System (SIPS) for processing data from the Ozone Monitoring Instrument (OMI) which will launch on the Aura platform in mid 2004. The Ozone Monitoring Instrument (OMI) is a contribution of the Netherlands Agency for Aerospace Programs (NIVR) in collaboration with the Finnish Meteorological Institute (FMI) to the Earth Observing System (EOS) Aura mission. It will continue the Total Ozone Monitoring System (TOMS) record for total ozone and other atmospheric parameters related to ozone chemistry and climate. OMI measurements will be highly synergistic with the other instruments on the EOS Aura platform. The LTP previously developed the Moderate Resolution Imaging Spectrometer (MODIS) Data Processing System (MODAPS), which has been in full operations since the launches of the Terra and Aqua spacecrafts in December, 1999 and May, 2002 respectively. During that time, it has continually evolved to better support the needs of the MODIS team. We now run multiple instances of the system managing faster than real time reprocessings of the data as well as continuing forward processing. The new OMI Data Processing System (OMIDAPS) was adapted from the MODAPS. It will ingest raw data from the satellite ground station and process it to produce calibrated, geolocated higher level data products. These data products will be transmitted to the Goddard Distributed Active Archive Center (GDAAC) instance of the Earth Observing System (EOS) Data and Information System (EOSDIS) for long term archive and distribution to the public. The OMIDAPS will also provide data distribution to the OMI Science Team for quality assessment, algorithm improvement, calibration, etc. We have taken advantage of lessons learned from the MODIS experience and software already developed for MODIS. We made some changes in the hardware system organization, database and software to adapt the system for OMI. We replaced the fundamental database system, Sybase, with an Open Source RDBMS called PostgreSQL, and based the entire OMIDAPS on a cluster of Linux based commodity computers rather than the large SGI servers that MODAPS uses. Rather than relying on a central I/O server host, the new system distributes its data archive among multiple server hosts in the cluster. OMI is also customizing the graphical user interfaces and reporting structure to more closely meet the needs of the OMI Science Team. Prior to 2003, simulated OMI data and the science algorithms were not ready for production testing. We initially constructed a prototype system and tested using a 25 year dataset of Total Ozone Mapping Spectrometer (TOMS) and Solar Backscatter Ultraviolet Instrument (SBUV) data. This prototype system provided a platform to support the adaptation of the algorithms for OMI, and provided reprocessing of the historical data aiding in its analysis. In a recent reanalysis of the TOMS data, the OMIDAPS processed 108,000 full orbits of data through 4 processing steps per orbit, producing about 800,000 files (400 GiB) of level 2 and greater data files. More recently we have installed two instances of the OMIDAPS for integration and testing of OM1 science processes as they get delivered from the Science Team. A Test instance of the OMIDAPS has also supported a series of "Interface Confidence Tests" (ICTs) and End-to-End Ground System tests to ensure the launch readiness of the system. This paper will discuss the high-level hardware, software, and database organization of the OMIDAPS and how it builds on the MODAPS heritage system. It will also provide an overview of the testing and implementation of the production OMIDAPS.

Tilmes, Curt↗
DOE OSTI2024

Joint Management and Optimization of Residential Natural Gas and Electricity Distribution Networks Coupled via Fuel Cells

The interesting properties of natural gas as well as the growing electric power demand worldwide have led to increasing attention to natural-gas-based distributed generation applications in electric distribution systems. This paper goes over the interdependency between a residential natural gas network and an electric distribution network that are coupled via fuel cells. The modeling of the gas network is introduced first, and then the algorithm for gas flow study is presented. The optimal placement and sizing of fuel cell based distributed generation systems are formulated to minimize the losses in both the gas and electric distribution networks, subject to their model constraints. In addition to this, in order to capture the probabilistic nature of the optimization problem under study, the K-means clustering algorithm is applied to the gas and electricity demands to determine hourly load states and their corresponding probabilities. Furthermore, simulation studies are carried out on an integrated system consisting of the IEEE 69-bus distribution feeder and a radial 27-node natural gas network to verify the developed optimization model and the proposed method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Distributed Berkeley Efficient Long-Read to Long-Read Aligner and Overlapper (DiBELLA) v1.0.0

We present a parallel algorithm and scalable implementation for genome analysis, specifically the problem of finding overlaps and alignments for data from "third generation" long read sequencers. While long sequences of DNA offer enormous advantages for biological analysis and insight, current long read sequencing instruments have high error rates and therefore require different approaches to analysis than their short read counterparts. Our work focuses on an efficient distributed-memory parallelization of an accurate single-node algorithm for overlapping and aligning long reads. We achieve scalability of this irregular algorithm by addressing the competing issues of increasing parallelism, minimizing communication, constraining the memory footprint, and ensuring good load balance. The resulting application, DiBELLA, is the first distributed memory overlapper and aligner specifically designed for long reads and parallel scalability.

Ellis, Marquita↗
DOE OSTI2020

Distribution Feeder-Scale Fast Frequency Response via Optimal Coordination of Net-load Resources Part II: Large-Scale Demonstration

This work is the second of a two-part series in which we develop and experimentally demonstrate a hierarchical control solution for optimally coordinating thousands of deferrable loads and distributed energy resources (DERs) to provide fast frequency response (FFR) from an entire distribution feeder. In Part I, we developed and proved practical algorithms for fast, cost-based optimal dispatch and for determining the optimal amount of headroom to operate solar inverters with to support FFR dispatch while minimizing opportunity cost. Simulation results in Part I demonstrated the advantages of the hierarchical dispatch approach in being able to maintain fast solution times needed for FFR even when the problem size increases. In Part II, we implement the algorithms developed in Part I in a novel, large-scale power hardware-in-the-loop experiment including embedded controllers and more than 100 powered appliance loads and DER connected to a simulated real-world distribution system with more than 10,000 controlled devices. Experimental results from multiple scenarios confirm that the optimal FFR dispatch approach scales well and can optimally coordinate more than 10,000 net-load resources across a distribution network while achieving hardware response times within 500 ms, which is not possible using state-of-the-art optimal coordination approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1990

Use of microwave satellite data to study variations in rainfall over the Indian Ocean

The University of Wisconsin Space Science and Engineering Center mapped rainfall over the Indian Ocean using a newly developed Scanning Multichannel Microwave Radiometer (SMMR) rain-retrieval algorithm. The short-range objective was to characterize the distribution and variability of Indian Ocean rainfall on seasonal and annual scales. In the long-range, the objective is to clarify differences between land and marine regimes of monsoon rain. Researchers developed a semi-empirical algorithm for retrieving Indian Ocean rainfall. Tools for this development have come from radiative transfer and cloud liquid water models. Where possible, ground truth information from available radars was used in development and testing. SMMR rainfalls were also compared with Indian Ocean gauge rainfalls. Final Indian Ocean maps were produced for months, seasons, and years and interpreted in terms of historical analysis over the sub-continent.

Hinton, Barry B.↗
NASA NTRS2001

Ground-Based Passive Microwave Remote Sensing Observations of Soil Moisture at S and L Band with Insight into Measurement Accuracy

Accurate estimates of spatially heterogeneous algorithm variables and parameters are required in determining the spatial distribution of soil moisture using radiometer data from aircraft and satellites. A ground-based experiment in passive microwave remote sensing of soil moisture was conducted in Huntsville, Alabama from July 1-14, 1996 to study retrieval algorithms and their sensitivity to variable and parameter specification. With high temporal frequency observations at S and L band, we were able to observe large scale moisture changes following irrigation and rainfall events, as well as diurnal behavior of surface moisture among three plots, one bare, one covered with short grass and another covered with alfalfa. The L band emitting depth was determined to be on the order of 0-3 or 0-5 cm below 0.30 cubic centimeter/cubic centimeter with an indication of a shallower emitting depth at higher moisture values. Surface moisture behavior was less apparent on the vegetated plots than it was on the bare plot because there was less moisture gradient and because of difficulty in determining vegetation water content and estimating the vegetation b parameter. Discrepancies between remotely sensed and gravimetric, soil moisture estimates on the vegetated plots point to an incomplete understanding of the requirements needed to correct for the effects of vegetation attenuation. Quantifying the uncertainty in moisture estimates is vital if applications are to utilize remotely-sensed soil moisture data. Computations based only on the real part of the complex dielectric constant and/or an alternative dielectric mixing model contribute a relatively insignificant amount of uncertainty to estimates of soil moisture. Rather, the retrieval algorithm is much more sensitive to soil properties, surface roughness and biomass.

Laymon, Charles A.↗
DOE OSTI2024

Efficient First-Order Algorithms for Large-Scale, Non-Smooth Maximum Entropy Models with Application to Wildfire Science

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback–Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O(mn) operations and can be trivially parallelized. Moreover, the strong ℓ1 convexity of the Kullback–Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Physics↗
NASA NTRS2016

Design, Development, and Testing of a UAV Hardware-in-the-Loop Testbed for Aviation and Airspace Prognostics Research

The airspace is becoming more and more complicated, and will continue to do so in the future with the integration of Unmanned Aerial Vehicles (UAVs), autonomy, spacecraft, other forms of aviation technology into the airspace. The new technology and complexity increases the importance and difficulty of safety assurance. Additionally, testing new technologies on complex aviation systems & systems of systems can be very difficult, expensive, and sometimes unsafe in real life scenarios. Prognostic methodology provides an estimate of the health and risks of a component, vehicle, or airspace and knowledge of how that will change over time. That measure is especially useful in safety determination, mission planning, and maintenance scheduling. The developed testbed will be used to validate prediction algorithms for the real-time safety monitoring of the National Airspace System (NAS) and the prediction of unsafe events. The framework injects flight related anomalies related to ground systems, routing, airport congestion, etc. to test and verify algorithms for NAS safety. In our research work, we develop a live, distributed, hardware-in-the-loop testbed for aviation and airspace prognostics along with exploring further research possibilities to verify and validate future algorithms for NAS safety. The testbed integrates virtual aircraft using the X-Plane simulator and X-PlaneConnect toolbox, UAVs using onboard sensors and cellular communications, and hardware in the loop components. In addition, the testbed includes an additional research framework to support and simplify future research activities. It enables safe, accurate, and inexpensive experimentation and research into airspace and vehicle prognosis that would not have been possible otherwise. This paper describes the design, development, and testing of this system. Software reliability, safety and latency are some of the critical design considerations in development of the testbed. Integration of HITL elements in the development phases and veri cation/ validation are key elements to this report.

HITL↗
NASA NTRS2010

Analyzing Non Stationary Processes in Radiometers

The lack of well-developed techniques for modeling changing statistical moments in our observations has stymied the application of stochastic process theory for many scientific and engineering applications. Non linear effects of the observation methodology is one of the most perplexing aspects to modeling non stationary processes. This perplexing problem was encountered when modeling the effect of non stationary receiver fluctuations on the performance of radiometer calibration architectures. Existing modeling approaches were found not applicable; particularly problematic is modeling processes across scales over which they begin to exhibit non stationary behavior within the time interval of the calibration algorithm. Alternatively, the radiometer output is modeled as samples from a sequence random variables; the random variables are treated using a conditional probability distribution function conditioned on the use of the variable in the calibration algorithm. This approach of treating a process as a sequence of random variables with non stationary stochastic moments produce sensible predictions of temporal effects of calibration algorithms. To test these model predictions, an experiment using the Millimeter wave Imaging Radiometer (MIR) was conducted. The MIR with its two black body calibration references was configured in a laboratory setting to observe a third ultra-stable reference (CryoTarget). The MIR was programmed to sequentially sample each of the three references in approximately a 1 second cycle. Data were collected over a six-hour interval. The sequence of reference measurements form an ensemble sample set comprised of a series of three reference measurements. Two references are required to estimate the receiver response. A third reference is used to estimate the uncertainty in the estimate. Typically, calibration algorithms are designed to suppress the non stationary effects of receiver fluctuations. By treating the data sequence as an ensemble collection, it is possible to apply temporal algorithms which exacerbate the non stationary effects. By varying the algorithm, information about the properties of the non stationary receiver fluctuations is obtained. Comparisons of analytical calculations and statistical analysis of data demonstrate impressive agreement.

Racette, Paul↗
NASA NTRS2002

Effects of Uncertainty in TRMM Precipitation Radar Path Integrated Attenuation on Interannual Variations of Tropical Oceanic Rainfall

Considerable uncertainty surrounds the issue of whether precipitation over the tropical oceans (30 deg N/S) systematically changes with interannual sea-surface temperature (SST) anomalies that accompany El Nino (warm) and La Nina (cold) events. Time series of rainfall estimates from the Tropical Rainfall Measuring Mission (TRMM Precipitation Radar (PR) over the tropical oceans show marked differences with estimates from two TRMM Microwave Imager (TMI) passive microwave algorithms. We show that path-integrated attenuation derived from the effects of precipitation on the radar return from the ocean surface exhibits interannual variability that agrees closely with the TMI time series. Further analysis of the frequency distribution of PR (2A25 product) rain rates suggests that the algorithm incorporates the attenuation measurement in a very conservative fashion so as to optimize the instantaneous rain rates. Such an optimization appears to come at the expense of monitoring interannual climate variability.

Robertson, Franklin R.↗
DOE OSTI2025

Intrepid MCMC: Metropolis-Hastings with exploration

In engineering examples, one often encounters the need to sample from unnormalized distributions with complex shapes that may also be implicitly defined through a physical or numerical simulation model, making it computationally expensive to evaluate the associated density function. For such cases, MCMC has proven to be an invaluable tool. Random-walk Metropolis Methods (also known as Metropolis-Hastings (MH)), in particular, are highly popular for their simplicity, flexibility, and ease of implementation. However, most MH algorithms suffer from significant limitations when attempting to sample from distributions with multiple modes (particularly disconnected ones). Here, in this paper, we present Intrepid MCMC - a novel MH scheme that utilizes a simple coordinate transformation to significantly improve the mode-finding ability and convergence rate to the target distribution of random-walk Markov chains while retaining most of the simplicity of the vanilla MH paradigm. Through multiple examples, we showcase the improvement in the performance of Intrepid MCMC over vanilla MH for a wide variety of target distribution shapes. We also provide an analysis of the mixing behavior of the Intrepid Markov chain, as well as the efficiency of our algorithm for increasing dimensions. A thorough discussion is presented on the practical implementation of the Intrepid MCMC algorithm. Finally, its utility is highlighted through a Bayesian parameter inference problem for a two-degree-of-freedom oscillator under free vibration.

97 - MATHEMATICS AND COMPUTING↗
NASA NTRS1989

Simplified calculation of solar flux distribution on the side wall of cylindrical cavity solar receivers

A simple solar flux calculation algorithm for a cylindrical cavity type solar receiver has been developed and implemented on an IBM PC-AT. Using cone optics, the contour error method is utilized to handle the slope error of a paraboloidal concentrator. The flux distribution on the side wall is calculated by integration of the energy incident from cones emanating from all the differential elements on the concentrator. The calculations are done for any set of dimensions and properties of the receiver and the concentrator, and account for any spillover on the aperture plate. The results of this algorithm compared excellently with those predicted by more complicated programs. Because of the utilization of axial symmetry and overall simplification, it is extremely fast. It can be esily extended to other axisymmetric receiver geometries.

Bhandari, P.↗
NASA NTRS1993

Lossless compression of image data products on th e FIFE CD-ROM series

How do you store enough of the key data sets, from a total of 120 gigabytes of data collected for a scientific experiment, on a collection of CD-ROM's, small enough to distribute to a broad scientific community? In such an application where information loss in unacceptable, lossless compression algorithms are the only choice. Although lossy compression algorithms can provide an order of magnitude improvement in compression ratios over lossless algorithms the information that is lost is often part of the key scientific precision of the data. Therefore, lossless compression algorithms are and will continue to be extremely important in minimizing archiving storage requirements and distribution of large earth and space (ESS) data sets while preserving the essential scientific precision of the data.

Newcomer, Jeffrey A.↗
DOE OSTI2020

Structural evolution of two-dimensional silicates using a “bond-switching” algorithm

Silicates are the most abundant materials in the earth's crust. In recent years, two-dimensional (2D) versions of them grown on metal supports (known as bilayer silicates) have allowed their study in detail down to the atomic scale. These structures are self-containing. Furthermore, they are not covalently bound to the metal support but interact with it through van der Waals forces. Like their three-dimensional counterparts, the 2D-silicates can form both crystalline and vitreous structures. Furthermore, the interconversion between vitreous to crystalline structures has been experimentally observed at the nanoscale. While theoretical work has been carried out to try to understand these transformations, a limitation for ab initio methods, and even molecular dynamics methods, is the computational cost of studying large systems and long timescales. In this work, we present a simple and computationally inexpensive approach, that can be used to represent the evolution of bilayer silicates using a bond-switching algorithm. This approach allows reaching equilibrium ring size distributions as a function of a parameter that can be related to the ratio between temperature and the energy required for the bond-switching event. The ring size distributions are compared to experimental data available in the literature.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

A fast particle-mesh simulation of non-linear cosmological structure formation with massive neutrinos

Quasi-N-body simulations, such as FastPM, provide a fast way to simulate cosmological structure formation, but have yet to adequately include the effects of massive neutrinos. In this work, we present a method to include neutrino particles in FastPM, enabling computation of the CDM and total matter power spectra to percent-level accuracy in the non-linear regime. The CDM-neutrino cross-power can also be computed at a sufficient accuracy to constrain cosmological observables. To avoid the shot noise that typically plagues neutrino particle simulations, we employ a quasi-random algorithm to sample the relevant Fermi-Dirac distribution when setting the initial neutrino thermal velocities. We additionally develop an effective distribution function to describe a set of non-degenerate neutrinos as a single particle to speed up non-degenerate simulations. The simulation is accurate for the full range of physical interest, M ν ≲ 0.6eV, and applicable to redshifts z ≲ 2. Such accuracy can be achieved by initializing particles with the two-fluid approximation transfer functions (using the REPS package). Convergence can be reached in ~ 25 steps, with a starting redshift of z=99. Probing progressively smaller scales only requires an increase in the number of CDM particles being simulated, while the number of neutrino particles can remain fixed at a value less than or similar to the number of CDM particles. In turn, the percentage increase in runtime-per-step due to neutrino particles is between ~ 5-20% for runs with 1024 3 CDM particles, and decreases as the number of CDM particles is increased. The code has been made publicly available, providing an invaluable resource to produce fast predictions for cosmological surveys and studying reconstruction.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

‘Flux+Mutability’: a conditional generative approach to one-class classification and anomaly detection

Abstract Anomaly Detection is becoming increasingly popular within the experimental physics community. At experiments such as the Large Hadron Collider, anomaly detection is growing in interest for finding new physics beyond the Standard Model. This paper details the implementation of a novel Machine Learning architecture, called Flux+Mutability, which combines cutting-edge conditional generative models with clustering algorithms. In the ‘flux’ stage we learn the distribution of a reference class. The ‘mutability’ stage at inference addresses if data significantly deviates from the reference class. We demonstrate the validity of our approach and its connection to multiple problems spanning from one-class classification to anomaly detection. In particular, we apply our method to the isolation of neutral showers in an electromagnetic calorimeter and show its performance in detecting anomalous dijets events from standard QCD background. This approach limits assumptions on the reference sample and remains agnostic to the complementary class of objects of a given problem. We describe the possibility of dynamically generating a reference population and defining selection criteria via quantile cuts. Remarkably this flexible architecture can be deployed for a wide range of problems, and applications like multi-class classification or data quality control are left for further exploration.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Non-Gaussian estimates of tensions in cosmological parameters

We discuss how to efficiently and reliably estimate the level of agreement and disagreement on parameter determinations from different experiments, fully taking into account non-Gaussianities in the parameter posteriors. We develop two families of scalable algorithms that allow us to perform this type of calculations in increasing number of dimensions and for different levels of tensions. One family of algorithms rely on kernel density estimates of posterior distributions while the other relies on machine learning modeling of the posterior distribution with normalizing flows. We showcase their effectiveness and accuracy with a set of benchmark examples and find both methods agree with each other and the true tension within 0.5σ in difficult cases and generally to 0.2σ or better. This allows us to study the level of internal agreement between different measurements of the clustering of cosmological structures from the Dark Energy Survey and their agreement with measurements of the Cosmic Microwave Background from the Planck satellite.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Feeder Power Disaggregation: A Data-Efficient Matrix Completion Approach

This paper presents a data-driven algorithm for the feeder power disaggregation problem in distribution systems. Leveraging spatio-temporal power patterns in residential homes, residential power is discomposed into three components: sparse-switching loads, periodic loads, and photovoltaic (PV) generation, which are characterized through the design of two sparse matrices and a low-rank matrix. The matrix completion process is data-efficient because of the matrix sparsity and low rankness, along with the use of power system models. The proposed approach is tested using real-world residential data set on a 33-bus distribution system, demonstrating accurate power disaggregation with efficient matrix completion.

distribution system↗