Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “clustering algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

The eROSITA Final Equatorial-Depth Survey (eFEDS)

Aims. The eROSITA Final Equatorial-Depth Survey (eFEDS), executed during the performance verification phase of the Spectrum-Roentgen-Gamma (SRG)/eROSITA telescope, was completed in November 2019. One of the science goals of this survey is to demonstrate the ability of eROSITA to detect samples of clusters and groups at the final depth of the eROSITA all-sky survey. Methods. Because of the sizeable (≈26" HEW FOV average) point-spread function of eROSITA, high-redshift clusters of galaxies or compact nearby groups hosting bright active galactic nuclei (AGN) can be misclassified as point sources by the source detection algorithms. A total of 346 galaxy clusters and groups in the redshift range of 0.1 < z < 1.3 were identified based on their red sequenc in the eFEDS point source catalog. Results. We examine the multiwavelength properties of these clusters and groups to understand the potential biases in our selection process and the completeness of the extent-selected sample. We find that the majority of the clusters and groups in the point source sample are indeed underluminous and compact compared to the extent-selected sample. Their faint X-ray emission, well below the flux limit of the extent-selected eFEDS clusters, and their compact X-ray emission are likely to be the main reason for this misclassification. In the sample, we confirm that 10% of the sources host AGN in their brightest cluster galaxies (BCGs) through optical spectroscopy and visual inspection. By studying their X-ray, optical, infrared, and radio properties, we establish a method for identifying clusters and groups that host AGN in their BCGs. We successfully test this method on the current point source catalog through the Sloan Digital Sky Survey optical spectroscopy and find eight low-mass clusters and groups with active radio-loud AGN that are particularly bright in the infrared. They include eFEDS J091437.8+024558, eFEDS J083520.1+012516, and eFEDS J092227.1+043339 at redshifts 0.3-0.4. Conclusions. This study helps us to characterize and understand our selection process and assess the completeness of the eROSITA extent-selected samples. The method we developed will be used to identify high-redshift clusters, AGN-dominated groups, and low-mass clusters that are misclassified in the future eROSITA all-sky survey point source catalogs.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2006

Probabilistic Classification Using Elemental Abundance Distributions and Lossless Image Compression in Apollo 17 Lunar Dust Samples from Mare Serenitatis

We have previously outlined a strategy for the detection of fossils [Storrie-Lombardi and Hoover, 2004] and extant microbial life [Storrie-Lombaudi and Hoover, 20051 during robotic missions to Mars using co-registered structural and chemical signatures. Data inputs included image lossless compression indices to estimate relative textural complexity and elemental abundance distributions. Two exploratory classification algorithms (principal component analysis and hierarchical cluster analysis) provide an initial tentative classification of all targets. Nonlinear stochastic neural networks are then trained to produce a Bayesian estimate of algorithm classification accuracy. The strategy previously has been successful in distinguishing regions of biotic and abiotic alteration of basalt glass from unaltered samples. [Storrie-Lombardi and Fisk, 2004; Storrie-Lombardi and Fisk, 2004] Such investigations of abiotic versus biotic alteration of terrestrial mineralogy on Earth are compromised by .the difficulty finding mineralogy completely unaffected by the ubiquitous presence of microbial life on the planet. The renewed interest in lunar exploration offers an opportunity to investigate geological materials that may exhibit signs of aqueous alteration, but are highly unlikely to contain contaminating biological weathering signatures. We here present an extension of our earlier data set to include lunar dust samples obtained during the Apollo 17 mission. Apollo 17 landed in the Taurus-Littrow Valley in Mare Serenitatis. Most of the rock samples from this region of the lunar highlands are basalts comprised primarily of plagioclase and pyroxene and selected examples of orange and black volcanic glass. SEM images and elemental abundances (C6, N7, O8, Na11, Mg12, Al13, Si14, P15, S16, Cll7, K19, Ca20, Fe26) for a series of targets in the lunar dust samples are compared to the extant cyanobacteria, fossil trilobites, Orgueil meteorite, and terrestrial basalt targets previously discussed. The data set provides a first step in producing a quantitative probabilistic methodology for geobiological analysis of returned lunar samples or in situ exploration.

Storrie-Lombardi, Michael C.↗
NASA NTRS2010

Mimas: Preliminary Evidence For Amorphous Water Ice from VIMS

We have conducted a statistical clustering analysis (1,2) on a mosaic of VIMS data cubes obtained on February 13, 2010, for Saturn s satellite Mimas. Seven VIMS cubes were geometrically projected and re-sampled to a common spatial resolution. The clustering technique consists of a partitioning algorithm coupled to a criterion that prevents sub-optimal solutions and tests for the influence of random noise in the measurements. The clustering technique is agnostic about the meaning of the clusters, and scientific interpretation requires their a posteriori evaluation. The preliminary results yielded five clusters, demonstrating that spectral variability across Mimas surface is statistically significant. The ratios of the means calculated for each of the clusters show structure within the 1.6- micron water ice band, as well as the shape and the central wavelength of the strong ice band at 2 micron, that map spatially in patterns apparently related to the topography of Mimas, in particular certain regions in and around Herschel crater. The mean spectra of the five clusters, show similarities with laboratory spectra of amorphous and crystalline H2O ice (3) that are suggestive of the presence of an amorphous ice component in certain regions of Mimas, notably on the central peak of Herschel, on the crater floor, and in faults surrounding the crater. This may represent a mixture of both ice phases, or perhaps a layer of amorphous ice on a base of crystalline ice. Another possible occurrence of amorphous ice appears southwest of Herschel, close to the south pole.

Cruikshank, Dale P.↗
NASA NTRS2016

Automated Grouping of Opportunity Rover Alpha Particle X-Ray Spectrometer Compositional Data

The Alpha Particle X-ray Spectrometer (APXS) conducts high-precision in situ measurements of rocks and soils on both active NASA Mars rovers. Since 2004 the rover Opportunity has acquired around 440 unique APXS measurements, including a wide variety of compositions, during its 42+ kilometers traverse across several geological formations. Here we discuss an analytical comparison algorithm providing a means to cluster samples due to compositional similarity and the resulting automated classification scheme. Due to the inherent variance of elements in the APXS data set, each element has an associated weight that is inversely proportional to the variance. Thus, the more consistent the abundance of an element in the data set, the more it contributes to the classification. All 16 elements standard to the APXS data set are considered. Careful attention is also given to the errors associated with the composition measured by the APXS - larger uncertainties reduce the weighting of the element accordingly. The comparison of two targets, i and j, generates a similarity score, S(sub ij). This score is immediately comparable to an average ratio across all elements if one assumes standard weighted uncertainty. The algorithm facilitates the classification of APXS targets by chemistry alone - independent of target appearance and geological context which can be added later as a consistency check. For the N targets considered, a N by N hollow matrix, S, is generated where S = S(sup T). The average relation score, S(sub av), for target N(sub i) is simply the average of column i of S. A large S(sub av) is indicative of a unique sample. In such an instance any targets with a low comparison score can be classified alike. The threshold between classes requires careful consideration. Applying the algorithm to recent Marathon Valley targets indicates similarities with Burns formation and average-Mars-like rocks encountered earlier at Endeavour Crater as well as a new class of felsic rocks.

VanBommel, S. J.↗
DOE OSTI2020

Modular performance prediction for scientific workflows using Machine Learning

Scientific workflows provide an opportunity for declarative computational experiment design in an intuitive and efficient way. A distributed workflow is typically executed on a variety of resources, and it uses a variety of computational algorithms or tools to achieve the desired outcomes. Such a variety imposes additional complexity in scheduling these workflows on large scale computers. As computation becomes more distributed, insights into expected workload that a workflow presents become critical for effective resource allocation. In this paper, we present a modular framework that leverages Machine Learning for creating precise performance predictions of a workflow. The central idea is to partition a workflow in such a way that makes the task of forecasting each atomic unit manageable and gives us a way to combine the individual predictions efficiently. We recognize a combination of an executable and a specific physical resource as a single module. This gives us a handle to characterize workload and machine power as a single unit of prediction. Overall, our modular technique of creating atomic modules and deployment of longest-path approach to estimate workflow performance, allows the framework to adapt to highly complex nested directed acyclic workflows and scale to new scenarios, since it does not make assumptions of underlying workflow structure. We present performance estimation results of independent workflow modules executed on the XSEDE SDSC Comet cluster using various Machine Learning algorithms. The results provide insights into the behavior and effectiveness of different algorithms in the context of scientific workflow performance prediction.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1980

ICAP: An Interactive Cluster Analysis Procedure for analyzing remotely sensed data

An Interactive Cluster Analysis Procedure (ICAP) was developed to derive classifier training statistics from remotely sensed data. The algorithm interfaces the rapid numerical processing capacity of a computer with the human ability to integrate qualitative information. Control of the clustering process alternates between the algorithm, which creates new centroids and forms clusters and the analyst, who evaluate and elect to modify the cluster structure. Clusters can be deleted or lumped pairwise, or new centroids can be added. A summary of the cluster statistics can be requested to facilitate cluster manipulation. The ICAP was implemented in APL (A Programming Language), an interactive computer language. The flexibility of the algorithm was evaluated using data from different LANDSAT scenes to simulate two situations: one in which the analyst is assumed to have no prior knowledge about the data and wishes to have the clusters formed more or less automatically; and the other in which the analyst is assumed to have some knowledge about the data structure and wishes to use that information to closely supervise the clustering process. For comparison, an existing clustering method was also applied to the two data sets.

Wharton, S. W.↗
NASA NTRS1996

Development of Collaborative Research Initiatives to Advance the Aerospace Sciences-via the Communications, Electronics, Information Systems Focus Group

The primary goal of the Adaptive Vision Laboratory Research project was to develop advanced computer vision systems for automatic target recognition. The approach used in this effort combined several machine learning paradigms including evolutionary learning algorithms, neural networks, and adaptive clustering techniques to develop the E-MOR.PH system. This system is capable of generating pattern recognition systems to solve a wide variety of complex recognition tasks. A series of simulation experiments were conducted using E-MORPH to solve problems in OCR, military target recognition, industrial inspection, and medical image analysis. The bulk of the funds provided through this grant were used to purchase computer hardware and software to support these computationally intensive simulations. The payoff from this effort is the reduced need for human involvement in the design and implementation of recognition systems. We have shown that the techniques used in E-MORPH are generic and readily transition to other problem domains. Specifically, E-MORPH is multi-phase evolutionary leaming system that evolves cooperative sets of features detectors and combines their response using an adaptive classifier to form a complete pattern recognition system. The system can operate on binary or grayscale images. In our most recent experiments, we used multi-resolution images that are formed by applying a Gabor wavelet transform to a set of grayscale input images. To begin the leaming process, candidate chips are extracted from the multi-resolution images to form a training set and a test set. A population of detector sets is randomly initialized to start the evolutionary process. Using a combination of evolutionary programming and genetic algorithms, the feature detectors are enhanced to solve a recognition problem. The design of E-MORPH and recognition results for a complex problem in medical image analysis are described at the end of this report. The specific task involves the identification of vertebrae in x-ray images of human spinal columns. This problem is extremely challenging because the individual vertebra exhibit variation in shape, scale, orientation, and contrast. E-MORPH generated several accurate recognition systems to solve this task. This dual use of this ATR technology clearly demonstrates the flexibility and power of our approach.

Knasel, T. Michael↗
DOE OSTI2022

Avoiding excess computation in asynchronous evolutionary algorithms

Abstract Asynchronous evolutionary algorithms are becoming increasingly popular as a means of making full use of many processors while solving computationally expensive search and optimization problems. These algorithms excel at keeping large clusters fully utilized, but may sometimes inefficiently sample an excess of fast‐evaluating solutions at the expense of higher‐quality, slow‐evaluating ones. We have previously introduced a steady‐state parent selection strategy, SWEET (“Selection whilE EvaluaTing”), that sometimes selects individuals that are still being evaluated and allows them to reproduce early. We perform a takeover‐time analysis that confirms that this strategy gives slow‐evaluating individuals that have higher fitnesses an increased ability to multiply in the population. We also find that SWEET appears effective at improving optimization performance on problems in which solution quality is positively correlated with evaluation time. We evaluate our approach on six simulated real‐valued optimization problems and three real‐world applications: an autonomous vehicle controller problem that involves tuning a spiking neural network and two adversarial EA problems. We further evaluate SWEET versus a basic asynchronous process in a simulated setting. We present evidence that SWEET outperforms basic asynchronous processes in a use‐case in which performance is positively correlated with evaluation time, and performs comparably (and often better) than basic asynchronous processes in several use‐cases where performance is negatively correlated with evaluation time. That said, in the cases where performance and evaluation time are negatively correlated the variance of outcomes for SWEET is notably high.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Avoiding Excess Computation in Asynchronous Evolutionary Algorithms

Asynchronous evolutionary algorithms are becoming increasingly popular as a means of making full use of many processors while solving computationally expensive search and optimization problems. These algorithms excel at keeping large clusters fully utilized, but may sometimes inefficiently sample an excess of fast-evaluating solutions at the expense of higher-quality, slow-evaluating ones. We introduce a steady-state parent selection strategy, SWEET (“Selection whilE EvaluaTing”), that sometimes selects individuals that are still being evaluated and allows them to reproduce early. This gives slow-evaluating individuals that have higher fitnesses an increased ability to multiply in the population. We find that SWEET appears effective in simulated take-over time analysis, but that its benefit is confined mostly to early in the run, and our preliminary study on an autonomous vehicle controller problem that involves tuning a spiking neural network proves inconclusive.

Scott, Eric↗
DOE OSTI2020

Distributed non-negative matrix factorization with determination of the number of latent features

The holistic analysis and understanding of the latent (that is, not directly observable) variables and patterns buried in large datasets is crucial for data-driven science, decision making and emergency response. Such exploratory analyses require devising unsupervised learning methods for data mining and extraction of the latent features, and non-negative matrix factorization (NMF) is one of the prominent such methods. NMF is based on compute-intense non-convex constrained minimization, which, for large datasets requires fast and distributed algorithms. However, current parallel implementations of NMF fail to estimate the number of latent features. In practice, identifying these features is both difficult and significant for pattern recognition and latent feature analysis, especially for large dense matrices. Here, we introduce a distributed NMF algorithm coupled with distributed custom clustering followed by a stability analysis on dense data, which we call DnMFk, to determine the number of latent variables. The results on synthetic data and the classical Swimmer data set demonstrate the accuracy of model determination while scaling nearly linearly across multiple processors for large data. Further, we employ DnMFk to determine the number of hidden features from a terabyte matrix.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Global Sensitivity Analysis of a Reactive Transport Model for Mineral Scale Formation During Hydraulic Fracturing

Injection of water-based hydraulic fracturing fluid (HFF) into tight shale gas/oil formations can increase formation permeability and enhance production rates, but this process frequently causes mineral scale formation that can occlude pore space and hinder flow. To identify the most important factors that control the formation of mineral scales, we applied a novel global sensitivity analysis method—distance-based generalized sensitivity analysis (DGSA)—to a reactive transport model (RTM) that was previously built and calibrated to simulate precipitation of barite [BaSO4] and iron (hydr)oxide [Fe(OH) 3 ] in shale matrices and on fracture surfaces. Reactive transport simulations were run with model parameters randomly sampled based on assigned uncertainties. Modeling results for barite and Fe(OH)3 formation were clustered using machine-learning algorithms. A list of ranked critical input parameters was obtained after statistical quantification of cumulative distribution functions of input parameters. We found that barite formation is most sensitive to the rate of sulfate ion generation, which is determined by the pyrite dissolution rate coefficient and oxidant availability. In addition, barite formation is sensitive to the initial amounts of barite in HFF and shale, followed by barite thermodynamics/kinetics. For Fe(OH) 3 formation, the ranked factors are Fe(OH)3 precipitation rate coefficients, initial HFF pH, initial Fe(OH) 3 amount in HFF, and oxidant availability. Overall, our results provide insights into managing mineral scale formation during hydraulic fracturing to enhance production. Meanwhile, this study serves as an example of global sensitivity analysis of RTMs using the efficient, straightforward, and open-source DGSA method.

58 GEOSCIENCES↗
DOE OSTI2023

An Extension of the Athena++ Code Framework for Radiation-magnetohydrodynamics in General Relativity Using a Finite-solid-angle Discretization

We extend the general-relativistic magnetohydrodynamics (GRMHD) capabilities of Athena++ to incorporate radiation. The intensity field in each finite-volume cell is discretized in angle, with explicit transport in both space and angle properly accounting for the effects of gravity on null geodesics, and with matter and radiation coupled in a locally implicit fashion. Here we describe the numerical procedure in detail, verifying its correctness with a suite of tests. Motivated in particular by black hole accretion in the high-accretion-rate, thin-disk regime, we demonstrate the application of the method to this problem. With excellent scaling on flagship computing clusters, the port of the algorithm to the GPU-enabled AthenaK code now allows the simulation of many previously intractable radiation-GRMHD systems.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1983

Synchronization and fault-masking in redundant real-time systems

A real time computer may fail because of massive component failures or not responding quickly enough to satisfy real time requirements. An increase in redundancy - a conventional means of improving reliability - can improve the former but can - in some cases - degrade the latter considerably due to the overhead associated with redundancy management, namely the time delay resulting from synchronization and voting/interactive consistency techniques. The implications of synchronization and voting/interactive consistency algorithms in N-modular clusters on reliability are considered. All these studies were carried out in the context of real time applications. As a demonstrative example, we have analyzed results from experiments conducted at the NASA Airlab on the Software Implemented Fault Tolerance (SIFT) computer. This analysis has indeed indicated that in most real time applications, it is better to employ hardware synchronization instead of software synchronization and not allow reconfiguration.

Krishna, C. M.↗
NASA NTRS2005

A Parallel Particle Swarm Optimization Algorithm Accelerated by Asynchronous Evaluations

A parallel Particle Swarm Optimization (PSO) algorithm is presented. Particle swarm optimization is a fairly recent addition to the family of non-gradient based, probabilistic search algorithms that is based on a simplified social model and is closely tied to swarming theory. Although PSO algorithms present several attractive properties to the designer, they are plagued by high computational cost as measured by elapsed time. One approach to reduce the elapsed time is to make use of coarse-grained parallelization to evaluate the design points. Previous parallel PSO algorithms were mostly implemented in a synchronous manner, where all design points within a design iteration are evaluated before the next iteration is started. This approach leads to poor parallel speedup in cases where a heterogeneous parallel environment is used and/or where the analysis time depends on the design point being analyzed. This paper introduces an asynchronous parallel PSO algorithm that greatly improves the parallel e ciency. The asynchronous algorithm is benchmarked on a cluster assembled of Apple Macintosh G5 desktop computers, using the multi-disciplinary optimization of a typical transport aircraft wing as an example.

Venter, Gerhard↗
NASA NTRS2006

PixelLearn

PixelLearn is an integrated user-interface computer program for classifying pixels in scientific images. Heretofore, training a machine-learning algorithm to classify pixels in images has been tedious and difficult. PixelLearn provides a graphical user interface that makes it faster and more intuitive, leading to more interactive exploration of image data sets. PixelLearn also provides image-enhancement controls to make it easier to see subtle details in images. PixelLearn opens images or sets of images in a variety of common scientific file formats and enables the user to interact with several supervised or unsupervised machine-learning pixel-classifying algorithms while the user continues to browse through the images. The machinelearning algorithms in PixelLearn use advanced clustering and classification methods that enable accuracy much higher than is achievable by most other software previously available for this purpose. PixelLearn is written in portable C++ and runs natively on computers running Linux, Windows, or Mac OS X.

Mazzoni, Dominic↗
DOE OSTI2025

Quantum many-body linear algebra, Hamiltonian moments, and a coupled-cluster inspired framework

Here, we propose a general strategy to develop quantum many-body approximations of primitives in linear algebra algorithms. As a practical example, we introduce a coupled-cluster inspired framework to produce approximate Hamiltonian moments and demonstrate its application in various linear algebra algorithms for ground state estimation. Through numerical examples, we illustrate the difference between the ground-state energies arising from quantum many-body linear algebra and those from the analogous many-body perturbation theory. Our results support the general idea of designing quantum many-body approximations outside of perturbation theory, providing a route to new algorithms and approximations.

Algorithms and data structure↗
DOE OSTI2021

A Massively Parallel Implementation of the CCSD(T) Method Using the Resolution-of-the-Identity Approximation and a Hybrid Distributed/Shared Memory Parallelization Model

In this work, a parallel algorithm is described for the coupled-cluster singles and doubles method augmented with a perturbative correction for triple excitations [CCSD(T)] using the resolution-of-the-identity (RI) approximation for two-electron repulsion integrals (ERIs). The algorithm bypasses the storage of four-center ERIs by adopting an integral-direct strategy. The CCSD amplitude equations are given in a compact quasi-linear form by factorizing them in terms of amplitude-dressed three-center intermediates. A hybrid MPI/OpenMP parallelization scheme is employed, which uses the OpenMP-based shared memory model for intranode parallelization and the MPI-based distributed memory model for internode parallelization. Parallel efficiency has been optimized for all terms in the CCSD amplitude equations. Two different algorithms have been implemented for the rate-limiting terms in the CCSD amplitude equations that entail and -scaling computational costs, where N O and N V denote the number of correlated occupied and virtual orbitals, respectively. One of the algorithms assembles the four-center ERIs requiring N V 4 and N O 2 N V 2 -scaling memory costs in a distributed manner on a number of MPI ranks, while the other algorithm completely bypasses the assembling of quartic memory-scaling ERIs and thus largely reduces the memory demand. It is demonstrated that the former memory-expensive algorithm is faster on a few hundred cores, while the latter memory-economic algorithm shows a better strong scaling in the limit of a few thousand cores. The program is shown to exhibit a near-linear scaling, in particular for the compute-intensive triples correction step, on up to 8000 cores. The performance of the program is demonstrated via calculations involving molecules with 24–51 atoms and up to 1624 atomic basis functions. As the first application, the complete basis set (CBS) limit for the interaction energy of the π-stacked uracil dimer from the S66 data set has been investigated. This work reports the first calculation of the interaction energy at the CCSD(T)/aug-cc-pVQZ level without local orbital approximation. The CBS limit for the CCSD correlation contribution to the interaction energy was found to be -8.01 kcal/mol, which agrees very well with the value -7.99 kcal/mol reported by Schmitz, Hättig, and Tew [ Phys. Chem. Chem. Phys. 2014 , 16 , 22167-22178]. The CBS limit for the total interaction energy was estimated to be -9.64 kcal/mol.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Linear-scaling quadruple excitations in local pair natural orbital coupled-cluster theory

Here, we present a fast, asymptotically linear-scaling implementation of the perturbative quadruples energy correction in coupled-cluster theory using local natural orbitals. Our work follows the domain-based local pair natural orbital (DLPNO) approach previously applied to lower levels of excitations in coupled-cluster theory. Our DLPNO-CCSDT(Q) algorithm uses converged doubles and triples amplitudes from a preceding DLPNO-CCSDT computation to compute the quadruples amplitude and energy in the quadruples natural orbital (QNO) basis. We demonstrate the compactness of the QNO space, showing that more than 95% of the (Q) correction can be recovered using relatively loose natural orbital cutoffs, compared to the tighter cutoffs used in pair and triples natural orbitals at lower levels of coupled-cluster theory. We also highlight the accuracy of our algorithm in the computation of relative energies, which yields deviations of sub-kJ mol −1 in relative energy compared to the canonical CCSDT(Q). Timings are conducted on a series of growing linear alkanes (up to 10 carbons and 608 basis functions) and water clusters (up to 49 water molecules and 2842 basis functions) to establish the asymptotic linear-scaling of our DLPNO-(Q) algorithm.

Auxiliary functions↗