Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,261 records · Page 70

Multilevel Monte Carlo Predictions of First Passage Times in Three-Dimensional Discrete Fracture Networks: A Graph-Based Approach

In this paper, we present a method combining multilevel Monte Carlo (MLMC) and a graph-based primary subnetwork identification algorithm to provide estimates of the mean and variance of the distribution of first passage times in fracture media at significantly lower computational cost than standard Monte Carlo (MC) methods. Simulations of solute transport are performed using a discrete fracture network (DFN), and instead of using various grid resolutions for levels in the MLMC, which is standard practice in MLMC, we identify a hierarchy of subnetworks in the DFN based on the shortest topological paths through the network using a graph-based method. While the mean of these ensembles is of critical importance, the variance is also essential in fractured media where uncertainty is an overarching theme, and understanding variability across an ensemble is a requirement for safety assessments. The method provides good estimates of the mean and variance at two orders of magnitude lower computational cost than MC.

54 ENVIRONMENTAL SCIENCES↗

Accelerating network layouts using graph neural networks

Graph layout algorithms used in network visualization represent the first and the most widely used tool to unveil the inner structure and the behavior of complex networks. Current network visualization software relies on the force-directed layout (FDL) algorithm, whose high computational complexity makes the visualization of large real networks computationally prohibitive and traps large graphs into high energy configurations, resulting in hard-to-interpret “hairball” layouts. Here we use Graph Neural Networks (GNN) to accelerate FDL, showing that deep learning can address both limitations of FDL: it offers a 10 to 100 fold improvement in speed while also yielding layouts which are more informative. We analytically derive the speedup offered by GNN, relating it to the number of outliers in the eigenspectrum of the adjacency matrix, predicting that GNNs are particularly effective for networks with communities and local regularities. Finally, we use GNN to generate a three-dimensional layout of the Internet, and introduce additional measures to assess the layout quality and its interpretability, exploring the algorithm’s ability to separate communities and the link-length distribution. The novel use of deep neural networks can help accelerate other network-based optimization problems as well, with applications from reaction-diffusion systems to epidemics.

97 MATHEMATICS AND COMPUTING↗

Variational quantum eigensolver with reduced circuit complexity

Abstract The variational quantum eigensolver (VQE) is one of the most promising algorithms to find eigenstates of a given Hamiltonian on noisy intermediate-scale quantum devices (NISQ). The practical realization is limited by the complexity of quantum circuits. Here we present an approach to reduce quantum circuit complexity in VQE for electronic structure calculations. Our ClusterVQE algorithm splits the initial qubit space into clusters which are further distributed on individual (shallower) quantum circuits. The clusters are obtained based on mutual information reflecting maximal entanglement between qubits, whereas inter-cluster correlation is taken into account via a new “dressed” Hamiltonian. ClusterVQE therefore allows exact simulation of the problem by using fewer qubits and shallower circuit depth at the cost of additional classical resources, making it a potential leader for quantum chemistry simulations on NISQ devices. Proof-of-principle demonstrations are presented for several molecular systems based on quantum simulators as well as IBM quantum devices.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

i- flow: High-dimensional integration and sampling with normalizing flows

In many fields of science, high-dimensional integration is required. Numerical methods have been developed to evaluate these complex integrals. We introduce the code i-flow, a python package that performs high-dimensional numerical integration utilizing normalizing flows. Normalizing flows are machine-learned, bijective mappings between two distributions. i-flow can also be used to sample random points according to complicated distributions in high dimensions. We compare i-flow to other algorithms for high-dimensional numerical integration and show that i-flow outperforms them for high dimensional correlated integrals. The i-flow code is publicly available on gitlab at https://gitlab.com/i-flow/i-flow.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

The miniJPAS survey quasar selection – II. Machine learning classification with photometric measurements and uncertainties

Astrophysical surveys rely heavily on the classification of sources as stars, galaxies, or quasars from multiband photometry. Surveys in narrow-band filters allow for greater discriminatory power, but the variety of different types and redshifts of the objects present a challenge to standard template-based methods. In this work, which is part of a larger effort that aims at building a catalogue of quasars from the miniJPAS survey, we present a machine learning-based method that employs convolutional neural networks (CNNs) to classify point-like sources including the information in the measurement errors. We validate our methods using data from the miniJPAS survey, a proof-of-concept project of the Javalambre Physics of the Accelerating Universe Astrophysical Survey (J-PAS) collaboration covering ∼1 deg 2 of the northern sky using the 56 narrow-band filters of the J-PAS survey. Due to the scarcity of real data, we trained our algorithms using mocks that were purpose-built to reproduce the distributions of different types of objects that we expect to find in the miniJPAS survey, as well as the properties of the real observations in terms of signal and noise. We compare the performance of the CNNs with other well-established machine learning classification methods based on decision trees, finding that the CNNs improve the classification when the measurement errors are provided as inputs. The predicted distribution of objects in miniJPAS is consistent with the putative luminosity functions of stars, quasars, and unresolved galaxies. Our results are a proof of concept for the idea that the J-PAS survey will be able to detect unprecedented numbers of quasars with high confidence.

79 ASTRONOMY AND ASTROPHYSICS↗

The Architecture of Metabolism Maximizes Biosynthetic Diversity in the Largest Class of Fungi

Abstract Ecological diversity in fungi is largely defined by metabolic traits, including the ability to produce secondary or “specialized” metabolites (SMs) that mediate interactions with other organisms. Fungal SM pathways are frequently encoded in biosynthetic gene clusters (BGCs), which facilitate the identification and characterization of metabolic pathways. Variation in BGC composition reflects the diversity of their SM products. Recent studies have documented surprising diversity of BGC repertoires among isolates of the same fungal species, yet little is known about how this population-level variation is inherited across macroevolutionary timescales. Here, we applied a novel linkage-based algorithm to reveal previously unexplored dimensions of diversity in BGC composition, distribution, and repertoire across 101 species of Dothideomycetes, which are considered the most phylogenetically diverse class of fungi and known to produce many SMs. We predicted both complementary and overlapping sets of clustered genes compared with existing methods and identified novel gene pairs that associate with known secondary metabolite genes. We found that variation among sets of BGCs in individual genomes is due to nonoverlapping BGC combinations and that several BGCs have biased ecological distributions, consistent with niche-specific selection. We observed that total BGC diversity scales linearly with increasing repertoire size, suggesting that secondary metabolites have little structural redundancy in individual fungi. We project that there is substantial unsampled BGC diversity across specific families of Dothideomycetes, which will provide a roadmap for future sampling efforts. Our approach and findings lend new insight into how BGC diversity is generated and maintained across an entire fungal taxonomic class.

59 BASIC BIOLOGICAL SCIENCES↗

True Load Balancing for Matricized Tensor Times Khatri-Rao Product

MTTKRP is the bottleneck operation in algorithms used to compute the CP tensor decomposition. For sparse tensors, utilizing the compressed sparse fibers (CSF) storage format and the CSF-oriented MTTKRP algorithms is important for both memory and computational efficiency on distributed-memory architectures. Existing intelligent tensor partitioning models assume the computational cost of MTTKRP to be proportional to the total number of nonzeros in the tensor. However, this is not the case for the CSF-oriented MTTKRP on distributed-memory architectures. We outline two deficiencies of nonzero-based intelligent partitioning models when CSF-oriented MTTKRP operations are performed locally: failure to encode processors' computational loads and increase in total computation due to fiber fragmentation. We focus on existing fine-grain hypergraph model and propose a novel vertex weighting scheme that enables this model encode correct computational loads of processors. We also propose to augment the fine-grain model by fiber nets for reducing the increase in total computational load via minimizing fiber fragmentation. In this way, the proposed model encodes minimizing the load of the bottleneck processor. In conclusion, parallel experiments with real-world sparse tensors on up to 1024 processors prove the validity of the outlined deficiencies and demonstrate the merit of our proposed improvements in terms of parallel runtimes.

97 MATHEMATICS AND COMPUTING↗

Automated nuclear cloud feature extraction from film

Chemical, biological, radiological, nuclear, and explosives incidents require rapid detection and characterization for appropriate response. For a nuclear detonation, visible-light cameras may be used to locate the cloud and characterize fallout deposition when coupled with numerical models. Films from the United States’ nuclear testing era compose the only sizeable collection of imagery depicting high-yield detonations. These films offer unique insights into characteristics of flows involving scales that are difficult to replicate experimentally, and they are a valuable source of data for the validation of models for nuclear fallout transport, either as part of emergency response or forensic activities. In this work, we implement modern computer vision and machine learning techniques to identify and track the cloud automatically and subsequently determine the time dependence of some of its features. We trained a ResNet-18 image classifier on hundreds of images to categorize nuclear cloud morphology. Each category or cloud regime is determined by early cloud evolution and is associated to constitutive properties of the flow, such as distribution of vorticity. Next, we identified keypoint features using the KAZE algorithm and tracked these keypoints in the images, allowing us to determine the dimensions and velocities of the cloud across film frames. These measurements converted to real-world units provide valuable experimental data that can be used in the development and validation of nuclear cloud models. We compared the results of this method against manual cloud rise measurements from two different films. In one, our automated method accelerated the feature extraction process without sacrificing measurement accuracy.

Khristy, Joel [ORNL] (ORCID:0000000209963060)↗

Interpreting machine learning prediction of fire emissions and comparison with FireMIP process-based models

Annual burned areas in the United States have increased 2-fold during the past decades. With more large fires resulting in more emissions of fine particulate matter, an accurate prediction of fire emissions is critical for quantifying the impacts of fires on air quality, human health, and climate. This study aims to construct a machine learning (ML) model with game-theory interpretation to predict monthly fire emissions over the contiguous US (CONUS) and to understand the controlling factors of fire emissions. The optimized ML model is used to diagnose the process-based models in the Fire Modeling Intercomparison Project (FireMIP) to inform future development. Results show promising performance for the ML model, Community Land Model (CLM), and Joint UK Land Environment Simulator-Interactive Fire And Emission Algorithm For Natural Environments (JULES-INFERNO) in reproducing the spatial distributions, seasonality, and interannual variability of fire emissions over the CONUS. Regional analysis shows that only the ML model and CLM simulate the realistic interannual variability of fire emissions for most of the subregions (r >0.95 for ML and r =0.14~0.70 for CLM), except for Mediterranean California, where all the models perform poorly (r =0.74 for ML and r <0.30 for the FireMIP models). Regarding seasonality, most models capture the peak emission in July over the western US. However, all models except for the ML model fail to reproduce the bimodal peaks in July and October over Mediterranean California, which may be explained by the smaller wind speeds of the atmospheric forcing data during Santa Ana wind events and limitations in model parameterizations for capturing the effects of Santa Ana winds on fire activity. Furthermore, most models struggle to capture the spring peak in emissions in the southeastern US, probably due to underrepresentation of human effects and the influences of winter dryness on fires in the models. As for extreme events, both the ML model and CLM successfully reproduce the frequency map of extreme emission occurrence but overestimate the number of months with extremely large fire emissions. Comparing the fire PM 2.5 emissions from the ML model with process-based fire models highlights their strengths and uncertainties for regional analysis and prediction and provides useful insights into future directions for model improvements.

54 ENVIRONMENTAL SCIENCES↗

Digital receiver simulation

Digital simulation of the data link for the Saturn-Uranus spacecraft design is summarized. The data link is a 40 watt, 400 MegaHertz, 44 bit-a-second link, a power starved link, and uses convolution coding. Aspects of the link of particular interest are atmospheric scintillation and the Doppler to data rate ratio. The major characteristics of the scintillation are modeled in terms of tapped delay lines. A candidate system design and receiver design is developed. A computer flow diagram depicting a routine for the error rate and one for acquisition is presented. Various coding algorithms are analyzed since the convolution code is sensitive to the distribution of error and the actual pattern of the error.

Hinrichs, C.↗

An investigation of the observability of ocean-surface parameters using GEOS-3 backscatter data

The degree to which ocean surface roughness can be synoptically observed through use of the information extracted from the GEOS-3 backscattered waveform data was evaluated. Algorithms are given for use in estimating the radar sensed waveheight distribution or ocean-surface impulse response. Other factors discussed include comparisons between theoretical and experimental radar cross section values, sea state bias effects, spatial variability of significant waveheight data, and sensor-related considerations.

Miller, L. S.↗

Development of mathematical techniques for the assimilation of remote sensing data into atmospheric models

The problem of the assimilation of remote sensing data into mathematical models of atmospheric pollutant species was investigated. The data assimilation problem is posed in terms of the matching of spatially integrated species burden measurements to the predicted three-dimensional concentration fields from atmospheric diffusion models. General conditions were derived for the reconstructability of atmospheric concentration distributions from data typical of remote sensing applications, and a computational algorithm (filter) for the processing of remote sensing data was developed.

Seinfeld, J. H.↗

Development of mathematical techniques for the assimilation of remote sensing data into atmospheric models

The problem of the assimilation of remote sensing data into mathematical models of atmospheric pollutant species was investigated. The problem is posed in terms of the matching of spatially integrated species burden measurements to the predicted three dimensional concentration fields from atmospheric diffusion models. General conditions are derived for the "reconstructability' of atmospheric concentration distributions from data typical of remote sensing applications, and a computational algorithm (filter) for the processing of remote sensing data is developed.

Seinfeld, J. H.↗

Scientific Requirements for Ozone Measurements

A study was conducted to determine the necessary parameters for the correlation of data on Earth ozone. First, requirements for the needed measurement accuracy for the change in ozone (yearly percent decrease) over a ten year period were determined. Secondly, the capabilities of the solar backscatter ultraviolet spectrometer instrumentation (SBUV) were examined. These included satellite coverage, radiometric calibration accuracy, instrument degradation, and algorithm accuracy. Third, and finally, ground based total and vertical zone distribution measurements, which could be used to support satellite observations, were considered.

Source record↗

The Raid distributed database system

Raid, a robust and adaptable distributed database system for transaction processing (TP), is described. Raid is a message-passing system, with server processes on each site to manage concurrent processing, consistent replicated copies during site failures, and atomic distributed commitment. A high-level layered communications package provides a clean location-independent interface between servers. The latest design of the package delivers messages via shared memory in a configuration with several servers linked into a single process. Raid provides the infrastructure to investigate various methods for supporting reliable distributed TP. Measurements on TP and server CPU time are presented, along with data from experiments on communications software, consistent replicated copy control during site failures, and concurrent distributed checkpointing. A software tool for evaluating the implementation of TP algorithms in an operating-system kernel is proposed.

Bhargava, Bharat↗

Deduction of emissions of source gases using an objective inversion algorithm and a chemical transport model

An inversion algorithm, constructed to deduce the emissions of a source gas required to produce a specified surface concentration, is applied to the observed surface concentrations of CFC 11, methylchloroform, and methane, using a two-dimensional chemical transport model. The information utilized for this deduction process is limited to the measured atmospheric concentration of the source gas, including the associated standard deviations of these measurements. In this way the amount of objective information available in these measurements is assessed. The algorithm is shown to be capable of producing a latitudinal emissions distribution as well as the error bounds on the deduced emission distribution. The 'ill posed' nature of this inverse problem is discussed as well as the implications this has on the spatial and temporal resolution at which emissions can be resolved. Finally, a methane emission distribution is deduced which has the expected seasonal variations and consistent with results from other, more subjective, deduction studies.

Brown, Margaret↗

Measurements of Ocean Features With AIRSAR

Here we discuss an indirect approach for measuring ocean features from synthetic aperture radar (SAR) imagery. The approach is based on the refraction of swell that can be seen in the SAR imagery. Locally, swell refraction is due only to changes in bathymetry and surface current shear and it is easy to separate these two effects. It has been shown that linear energetic shears can be detected and measured from high resolution images of the ocean. A simple approach to measuring surface velocity and shear is to divide the imagery into smaller segments where the shear in each segment is approximately linear. The known linear algorithm would then be applied to each segment and the velocity/shear distribution can then be determined by assembling the segments with linear shear into an estimate of the actual shear/velocity distribution.

Sheres, David↗

Luminous Binary Supersoft X-Ray Sources

We have made remarkable progress in the study of luminous supersoft X-ray sources during the past year. We have begun to discover a population of ultraluminous SSSs (e.g., in NGC 300 [Kong & Di Stefano 20031 as well as in Ml0l [Di Stefano & Kong 2003]), which may be accreting intermediate-mass (50-100 solar mass) black holes. This work follows from an algorithm we have developed (Di Stefano & Kong 2003) to identify SSSs in external galaxies, selecting them from among each galaxy s total population of X-ray sources. We have applied the algorithm to approximately one dozen galaxies and will make it public after it has been published in its entirety. Through our own application of the algorithm, we have discovered SSSs in every galaxy, mapping their spatial distribution, to obtain important clues to their fundamental natures. We have discovered that there is a large population of X-ray sources which are slightly hotter (100-250 eV) than standard SSSs. Some of these may be accreting BHs with masses between roughly 50 anf 100 solar masses. To explore this possibility, we are working on theoretical models for the formation and evolution of such systems (Di Stefano 2003).

Oliversen, Ronald J.↗