SEARCH · Engineering Papers

Results for “distributed clustering methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

NASA NTRS1994

Convergence rate enhancement of navier-stokes codes on clustered grids

Our Sensitivity-Based Minimal Residual (SBMR) method which is based on our earlier Distributed Minimal Residual (DMR) method allows each component of the solution vector in a system of equations to have its own convergence speed. Our global SBMR method was found to consistently outperform the DMR method while requiring considerably less computer memory. Recently, we have developed and tested a new Line SBMR or LSBMR method and a Time-Step-Scaling (TSS) method that are even more robust and computationally efficient than our global SBMR method, especially on highly clustered computational grids in laminar and turbulent flow computations.

Choi, Kwang-Yoon↗

DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗

NASA NTRS1974

An analysis of fracture trace patterns in areas of flat-lying sedimentary rocks for the detection of buried geologic structure

Two study areas in a cratonic platform underlain by flat-lying sedimentary rocks were analyzed to determine if a quantitative relationship exists between fracture trace patterns and their frequency distributions and subsurface structural closures which might contain petroleum. Fracture trace lengths and frequency (number of fracture traces per unit area) were analyzed by trend surface analysis and length frequency distributions also were compared to a standard Gaussian distribution. Composite rose diagrams of fracture traces were analyzed using a multivariate analysis method which grouped or clustered the rose diagrams and their respective areas on the basis of the behavior of the rays of the rose diagram. Analysis indicates that the lengths of fracture traces are log-normally distributed according to the mapping technique used. Fracture trace frequency appeared higher on the flanks of active structures and lower around passive reef structures. Fracture trace log-mean lengths were shorter over several types of structures, perhaps due to increased fracturing and subsequent erosion. Analysis of rose diagrams using a multivariate technique indicated lithology as the primary control for the lower grouping levels. Groupings at higher levels indicated that areas overlying active structures may be isolated from their neighbors by this technique while passive structures showed no differences which could be isolated.

Podwysocki, M. H.↗

DOE OSTI2023

A Two-Step Time-Series Data Clustering Method for Building-Level Load Profile

AMI↗

DOE OSTI2021

The Initial–Final Mass Relation for Hydrogen-deficient White Dwarfs

The initial–final mass relation represents the total mass lost by a star during the entirety of its evolution from the zero age main sequence to the white-dwarf cooling track. The semiempirical initial–final mass relation (IFMR) is largely based on observations of DA white dwarfs, the most common spectral type of white dwarf and the simplest atmosphere to model. We present a first derivation of the semiempirical IFMR for hydrogen-deficient (non-DA) white dwarfs in open star clusters. We identify a possible discrepancy between the DA and non-DA IFMRs, with non-DA white dwarfs ≈0.07 M {sub ⊙} less massive at a given initial mass. Such a discrepancy is unexpected based on theoretical models of non-DA formation and observations of field white dwarf mass distributions. If real, the discrepancy is likely due to enhanced mass loss during the final thermal pulse and renewed post-AGB evolution of the star. However, we are dubious that the mass discrepancy is physical and instead is due to the small sample size, to systematic issues in model atmospheres of non-DAs, and to the uncertain evolutionary history of Procyon B (spectral type DQZ). A significantly larger sample size is needed to test these assertions. In addition, we also present Monte Carlo models of the correlated errors for DA and non-DA white dwarfs in the initial–final mass plane. We find the uncertainties in initial–final mass determinations for individual white dwarfs can be significantly asymmetric, but the recovered functional form of the IFMR is grossly unaffected by the correlated errors.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

A Two-Step Time-Series Data Clustering Method for Building-Level Load Profile

Residential and commercial buildings have huge potential to contribute value to improve grid resilience by participating grid services. To reveal the significant value, it is critical to estimate the grid service capability from these buildings. Unlike the large-scale distributed energy resources such as wind and solar farms, those buildings need to participate grid services in aggregation, not by individual. Therefore, it is important to appropriately group buildings for aggregation. The load profiles in the same group will have similar characteristics at the same time step, so grid operators can send the grid service signal to the customer group with a higher chance to respond at that time step. In this paper, we develop a load profile clustering method to classify the building-level load profiles for grid service capability estimation. In our two-step clustering approach, we first calculate the total load consumption for each building, clustering the load profiles based on energy consumption level. Then, we further cluster the load profiles in each energy cluster based on the load shape. The parameter selection for each clustering step is discussed. The proposed method is applied on actual building-level load profiles, and the results have proved the effectiveness of this method.

advanced metering infrastructure (AMI)↗

NASA NTRS1979

CLASSY: An adaptive maximum likelihood clustering algorithm

The CLASSY clustering method alternates maximum likelihood iterative techniques for estimating the parameters of a mixture distribution with an adaptive procedure for splitting, combining, and eliminating the resultant components of the mixture. The adaptive procedure is based on maximizing the fit of a mixture of multivariate normal distributions to the observed data using its first through fourth central moments. It generates estimates of the number of multivariate normal components in the mixture as well as the proportion, mean vector, and covariance matrix for each component. The basic mathematical model for CLASSY and the actual operation of the algorithm as currently implemented are described. Results of applying CLASSY to real and simulated LANDSAT data are presented and compared with those generated by the iterative self-organizing clustering system algorithm on the same data sets.

Lennington, R. K.↗

DOE OSTI2024

MLEC-Sim: A Simulator for Evaluating Multi-Level Erasure Coding

We present MLEC-Sim, a sophisticated simulator for Multi-Level Erasure Coding (MLEC), developed in approximately 13 KLOC. The simulator is engineered to analyze the impact of various system configurations and erasure coding policies on system durability and network overhead. It supports a comprehensive range of parameters including disk capacity, disk I/O bandwidth, failure rates, network bandwidth, and system scale, accommodating various erasure coding approaches such as Single-Level Erasure Coding (SLEC), Multi-Level Erasure Coding (MLEC), and Local Reconstruction Codes (LRC). MLEC-Sim provides support for multiple chunk placement policies, including clustered parity and declustered parity, and encompasses a variety of repair methods like Repair-ALL, Repair-FCO, Repair-HYB, and Repair-MIN. It is capable of simulating disk failures through a variety of means, including distribution-based or trace-based mechanisms, and can handle complex multi-level (de)clustered placements and repair processes. A key feature of MLEC-Sim is its adoption of the splitting simulation method for evaluating system durabilities at extremely high levels, which are challenging to assess with traditional simulation approaches. This feature allows for a detailed evaluation of system resilience under a range of conditions, aiding in the selection of appropriate erasure coding solutions for enhancing system durability. MLEC-Sim contributes to the field of data storage and reliability by providing a tool for the detailed evaluation of the durability and efficiency of erasure coding configurations, intended for use by researchers and practitioners in the design and optimization of storage systems.

Wang, Meng↗

NASA NTRS2022

Rotation of Low-mass Stars in Upper Centaurus-Lupus and Lower Centaurus-Crux with TESS

We present stellar rotation rates derived from Transiting Exoplanet Survey Satellite (TESS) light curves for stars in Upper Centaurus–Lupus (UCL; ∼136 pc, ∼16 Myr) and Lower Centaurus–Crux (LCC; ∼115 pc, ∼17 Myr). We find spot-modulated periods (P) for ∼90% of members. The range of light-curve and periodogram shapes echoes that found for other clusters with K2, but fewer multiperiod stars may be an indication of the different noise characteristics of TESS, or a result of the source selection methods here. The distribution of P as a function of color as a proxy for mass fits nicely in between that for both older and younger clusters observed by K2, with fast rotators being found among both the highest and lowest masses probed here, and a well-organized distribution of M-star rotation rates. About 13% of the stars have an infrared excess, suggesting a circumstellar disk; this is well matched to expectations, given the age of the stars. There is an obvious pileup of disked M stars at P ∼ 2 days, and the pileup may move to shorter P as the mass decreases. There is also a strong concentration of disk-free M stars at P ∼ 2 days, hinting that perhaps these stars have recently freed themselves from their disks. Exploring the rotation rates of stars in UCL/LCC has the potential to help us understand the beginning of the end of the influence of disks on rotation, and the timescale on which stars respond to unlocking.

L M Rebull↗

NASA NTRS1993

An X-ray method for detecting substructure in galaxy clusters - Application to Perseus, A2256, Centaurus, Coma, and Sersic 40/6

We use the moments of the X-ray surface brightness distribution to constrain the dynamical state of a galaxy cluster. Using X-ray observations from the Einstein Observatory IPC, we measure the first moment FM, the ellipsoidal orientation angle, and the axial ratio at a sequence of radii in the cluster. We argue that a significant variation in the image centroid FM as a function of radius is evidence for a nonequilibrium feature in the intracluster medium (ICM) density distribution. In simple terms, centroid shifts indicate that the center of mass of the ICM varies with radius. This variation is a tracer of continuing dynamical evolution. For each cluster, we evaluate the significance of variations in the centroid of the IPC image by computing the same statistics on an ensemble of simulated cluster images. In producing these simulated images we include X-ray point source emission, telescope vignetting, Poisson noise, and characteristics of the IPC. Application of this new method to five Abell clusters reveals that the core of each one has significant substructure. In addition, we find significant variations in the orientation angle and the axial ratio for several of the clusters.

Mohr, Joseph J.↗

NASA NTRS1972

The estimation of masses of individual galaxies in clusters of galaxies.

Three different methods of estimating masses are discussed. The 'density method' is based on the analysis of the density distribution of galaxies around the object whose mass is to be found. The 'bound-galaxy method' gives estimates of the mass of a double, triple, or quadruple system from analysis of the orbital motion of the components. The 'virial method' utilizes the formulas derived for the second method to obtain estimates of the virial-theorem masses of whole clusters, and thus to obtain upper limits on the mass of an individual galaxy in a cluster. The analytic formulas are developed and compared with computer experiments, and some applications are given.

Wolf, R. A.↗

DOE OSTI2026

Joint Analysis of Small-scale Galaxy Clustering and Galaxy–Galaxy Lensing from BOSS Galaxies

We present a joint analysis of galaxy clustering and galaxy–galaxy lensing measurements from BOSS galaxies using a simulation-based emulation method combined with a halo occupation distribution model. Our emulators are constructed with the Aemulus ν simulations, a suite of wνCDM N-body simulations with massive neutrinos as independent particle species. We combine small-scale analysis of clustering from 0.1 to 60.2 h −1 Mpc and lensing from 1.7 to 60.2 h −1 Mpc to perform cosmological constraints. We split the BOSS galaxies into three redshift bins to measure their clustering and employ galaxies from Dark Energy Camera Legacy Survey and Hyper Suprime-Cam as source galaxies to measure lensing separately. We find that the addition of lensing significantly improves the constraining power on $S_8 = σ_8(Ω_m/0.3)^{0.5}$, with a weak improvement for fσ 8 . Our results of fσ 8 indicate tensions of around 1σ−4σ below the results of the cosmic microwave background observations of Planck. For S 8 , our results are also lower than Planck, and the tension can be mitigated when considering possible systematics in lensing measurement. As a by-product, our analysis prefers a nonzero neutrino mass but without strong significance, with the constraining power dominated by the clustering. Given the accuracy and precision of our model and the observational data, it is anticipated that larger and higher-quality spectroscopic data sets will improve the constraints on this fundamental property in the near future.

Gao, Wenhao [Shanghai Jiao Tong University (China)↗

NASA NTRS1978

A simple method for obtaining a three-dimensional proton distribution function from Voyager plasma data

The main sensor of the Vogager plasma experiment consists of a cluster of three, modulated-grid Faraday cups whose normals are arranged symmetrically about the symmetry axis of the cluster at an angle of 20 degrees to that axis. In interplanetary space, each cup explores the positive ion distribution by accepting particles from contiguous slices in velocity space. The slices are narrow in the direction of the normal to the modulating grid but are broad in planes parallel to that grid. The resulting three sets of measurements can be combined to yield the three-dimensional distribution function in the following way: the distribution function is assumed to be gyrotropic. For each value of speed in a frame of reference moving with the bulk velocity of the solar wind, the variation of the distribution function with angle from the field direction is represented by a series of Legendre polynomials. Effects such as double-streaming and heat flow can be well represented by using only the first three terms of the series which are fully specified by the measurements. Examples of the use of this method in the analysis of Voyager data are shown.

Olbert, S.↗

NASA NTRS2016

Improved Test Planning and Analysis Through the Use of Advanced Statistical Methods

The goal of this work is, through computational simulations, to provide statistically-based evidence to convince the testing community that a distributed testing approach is superior to a clustered testing approach for most situations. For clustered testing, numerous, repeated test points are acquired at a limited number of test conditions. For distributed testing, only one or a few test points are requested at many different conditions. The statistical techniques of Analysis of Variance (ANOVA), Design of Experiments (DOE) and Response Surface Methods (RSM) are applied to enable distributed test planning, data analysis and test augmentation. The D-Optimal class of DOE is used to plan an optimally efficient single- and multi-factor test. The resulting simulated test data are analyzed via ANOVA and a parametric model is constructed using RSM. Finally, ANOVA can be used to plan a second round of testing to augment the existing data set with new data points. The use of these techniques is demonstrated through several illustrative examples. To date, many thousands of comparisons have been performed and the results strongly support the conclusion that the distributed testing approach outperforms the clustered testing approach.

Green, Lawrence L.↗

DOE OSTI2021

Core Mass Estimates in Strong Lensing Galaxy Clusters Using a Single-halo Lens Model

The core mass of galaxy clusters is an important probe of structure formation. Here we evaluate the use of a single-halo model (SHM) as an efficient method to estimate the strong lensing cluster core mass, testing it with ray-traced images from the Outer Rim simulation. Unlike detailed lens models, the SHM represents the cluster mass distribution with a single halo and can be automatically generated from the measured lensing constraints. In this work we find that the projected core mass estimated with this method, $M_{\text{SHM}}$, has a scatter of 8.52% and a bias of 0.90% compared to the "true" mass within the same aperture. Our analysis shows no systematic correlation between the scatter or bias and the lens-source system properties. The bias and scatter can be reduced to 3.26% and 0.34%, respectively, by excluding models that fail a visual inspection test. We find that the SHM success depends on the lensing geometry, with single giant arc configurations accounting for most of the failed cases due to their limiting constraining power. When excluding such cases, we measure a scatter and bias of 3.88% and 0.84%, respectively. Finally, we find that when the source redshift is unknown, the model-predicted redshifts are overestimated, and the $M_{\text{SHM}}$ is underestimated by a few percent, highlighting the importance of securing spectroscopic redshifts of background sources. Our analysis provides a quantitative characterization of $M_{\text{SHM}}$, enabling its efficient use as a tool to estimate the strong lensing cluster core masses in the large samples, expected from current and future surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Sub-10 nm Probing of Ferroelectricity in Heterogeneous Materials by Machine Learning Enabled Contact Kelvin Probe Force Microscopy

Reducing the dimensions of ferroelectric materials down to the nanoscale has strong implications on the ferroelectric polarization pattern and on the ability to switch the polarization. As the size of ferroelectric domains shrinks to the nanometer scale, the heterogeneity of the polarization pattern becomes increasingly pronounced, enabling a large variety of possible polar textures in nanocrystalline and nanocomposite materials. Critical to the understanding of fundamental physics of such materials and hence their applications in electronic nanodevices is the ability to investigate their ferroelectric polarization at the nanoscale in a nondestructive way. We show that contact Kelvin probe force microscopy (cKPFM) combined with a k-means response clustering algorithm enables to measure the ferroelectric response at a mapping resolution of 8 nm. In a BaTiO 3 thin film on silicon composed of tetragonal and hexagonal nanocrystals, we determine a nanoscale lateral distribution of discrete ferroelectric response clusters, fully consistent with the nanostructure determined by transmission electron microscopy. Moreover, we apply this data clustering method to the cKPFM responses measured at different temperatures, which allows us to follow the corresponding change in the polarization pattern as the Curie temperature is approached and across the phase transition. This work opens up perspectives for mapping complex ferroelectric polarization textures such as curled/swirled polar textures that can be stabilized in epitaxial heterostructures and more generally for mapping the polar domain distribution of any spatially highly heterogeneous ferroelectric materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

ADDGALS: Simulated Sky Catalogs for Wide Field Galaxy Surveys

Abstract We present a method for creating simulated galaxy catalogs with realistic galaxy luminosities, broadband colors, and projected clustering over large cosmic volumes. The technique, denoted Addgals (Adding Density Dependent GAlaxies to Lightcone Simulations), uses an empirical approach to place galaxies within lightcone outputs of cosmological simulations. It can be applied to significantly lower-resolution simulations than those required for commonly used methods such as halo occupation distributions, subhalo abundance matching, and semi-analytic models, while still accurately reproducing projected galaxy clustering statistics down to scales of r ∼ 100 h −1 kpc . We show that Addgals catalogs reproduce several statistical properties of the galaxy distribution as measured by the Sloan Digital Sky Survey (SDSS) main galaxy sample, including galaxy number densities, observed magnitude and color distributions, as well as luminosity- and color-dependent clustering. We also compare to cluster–galaxy cross correlations, where we find significant discrepancies with measurements from SDSS that are likely linked to artificial subhalo disruption in the simulations. Applications of this model to simulations of deep wide-area photometric surveys, including modeling weak-lensing statistics, photometric redshifts, and galaxy cluster finding, are presented in DeRose et al., and an application to a full cosmology analysis of Dark Energy Survey (DES) Year 3 like data is presented in DeRose et al. We plan to publicly release a 10,313 square degree catalog constructed using Addgals with magnitudes appropriate for several existing and planned surveys, including SDSS, DES, VISTA, Wide-field Infrared Survey Explorer, and Rubin Observatory’s Legacy Survey of Space and Time.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Nonequilibrium information entropy approach to ternary fission of actinides

Ternary fission of actinides probes the state of the nucleus at scission. Light clusters are produced in space and time very close to the scission point. Within the nonequilibrium statistical operator method, a generalized Gibbs distribution is constructed from the information given by the observed yields of isotopes. Using this relevant statistical operator, yields are calculated taking excited states and continuum correlations into account, in accordance with the virial expansion of the equation of state. Furthermore, clusters with mass number A≤10 are well described using the nonequilibrium generalizations of temperature and chemical potentials. Improving the virial expansion, in-medium effects may become of importance in determining the contribution of weakly bound states and continuum correlations to the intrinsic partition function. Yields of larger clusters, which fail to reach this quasiequilibrium form of the relevant distribution, are described by nucleation kinetics, and a saddle-to-scission relaxation time of about 7000 fm/c is inferred. Light-charged particle emission, described by reaction kinetics and virial expansions, may therefore be regarded as a very important tool to probe the nonequilibrium time evolution of actinide nuclei during fission.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗