Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Detection of Anomalies in Gamma Background Radiation Data with K-Means and Self-Organizing Map Clustering Algorithms (Consortium on Nuclear Security Technologies (CONNECT) Q1 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. The challenge is that spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

61 RADIATION PROTECTION AND DOSIMETRY↗

Methodology for Clustering High-Resolution Spatiotemporal Solar Resource Data

In this report, we introduce a methodology to achieve multiple levels of spatial resolution reduction of solar resource data, with minimal impact on data variability, for use in energy systems modeling. The selection of an appropriate clustering algorithm, parameter selection including cluster size, methods of temporal data segmentation, and methods of cluster evaluation are explored in the context of a repeatable process. In describing this process, we illustrate the steps in creating a reduced resolution, but still viable, dataset to support energy systems modeling, e.g. capacity expansion or production cost modeling. This process is demonstrated through the use of a solar resource dataset; however, the methods are applicable to other resource data represented through spatiotemporal grids, including wind data. In addition to energy modeling, the techniques demonstrated in this paper can be used in a novel top-down approach to assess renewable resources within many other contexts that leverage variability in resource data but require reduction in spatial resolution to accommodate modeling or computing constraints.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Signature extension methods in crop area estimation

The Procedure B multispectral processing system is both multisegment and multistratum. It uses data from several LACIE-sized segments together and makes a proportion estimate for the entire group of segments as well as for the individual segments. In the clustering of data features, Procedure B produces multiple classes or strata rather than just two strata (as in Procedure 1), and performs stratified sampling on each of these mutliple strata in order to make a proportion estimate. Tests results for the components and for the overall performance of Procedure B are presented, and conclusions that can be drawn from these tests are discussed. The rationale for signature extension for crop area estimation is summarized.

Kauth, R. J.↗

Narrow-angle tail radio sources and evidence for radial orbits in Abell clusters

Published observational data on the tail orientations (TOs) of 60 narrow-angle-tail (NAT) radio sources in Abell clusters of galaxies are analyzed statistically using a maximum-likelihood approach. The results are presented in a table, and it is found that the observed TO distributions in the whole sample and in subsamples of morphologically regular NATs and NATs with pericentric distances d greater than 500 kpc are consistent with isotropic orbits, whereas the TOs for NATs with d less than 500 kpc are consistent with highly radial orbits. If radial orbits were observed near the centers of other types of cluster galaxies as well, it could be inferred that violent relaxation during cluster formation was incomplete, and that clusters form by spherical collapse and secondary infall, as proposed by Gunn (1977).

O'Dea, Christopher P.↗

Data-Driven Whole-Genome Clustering to Detect Geospatial, Temporal, and Functional Trends in SARS-CoV-2 Evolution

Current methods for defining SARS-CoV-2 lineages ignore the vast majority of the SARS-CoV-2 genome. We develop and apply an exhaustive vector comparison method that directly compares all known SARS-CoV-2 genome sequences to produce novel lineage classifications. We utilize data-driven models that (i) accurately capture the complex interactions across the set of all known SARS-CoV-2 genomes, (ii) scale to leadership-class computing systems, and (iii) enable tracking how such strains evolve geospatially over time. We show that during the height of the original Omicron surge, countries across Europe, Asia, and the Americas had a spatially asynchronous distribution of Omicron sub-strains. Moreover, neighboring countries were often dominated by either different clusters of the same variant or different variants altogether throughout the pandemic. Analyses of this kind may suggest a different pattern of epidemiological risk than was understood from conventional data, as well as produce actionable insights and transform our ability to prepare for and respond to current and future biological threats.

Jacobson, Daniel↗

Dark Energy Survey Year 1 Results: Multi-Probe Methodology and Simulated Likelihood Analyses

We present the methodology for and detail the implementation of the Dark Energy Survey (DES) 3x2pt DES Year 1 (Y1) analysis, which combines configuration-space two-point statistics from three different cosmological probes: cosmic shear, galaxy-galaxy lensing, and galaxy clustering, using data from the first year of DES observations. We have developed two independent modeling pipelines and describe the code validation process. We derive expressions for analytical real-space multi-probe covariances, and describe their validation with numerical simulations. We stress-test the inference pipelines in simulated likelihood analyses that vary 6-7 cosmology parameters plus 20 nuisance parameters and precisely resemble the analysis to be presented in the DES 3x2pt analysis paper, using a variety of simulated input data vectors with varying assumptions. We find that any disagreement between pipelines leads to changes in assigned likelihood $\Delta \chi^2 \le 0.045$ with respect to the statistical error of the DES Y1 data vector. We also find that angular binning and survey mask do not impact our analytic covariance at a significant level. We determine lower bounds on scales used for analysis of galaxy clustering (8 Mpc$~h^{-1}$) and galaxy-galaxy lensing (12 Mpc$~h^{-1}$) such that the impact of modeling uncertainties in the non-linear regime is well below statistical errors, and show that our analysis choices are robust against a variety of systematics. These tests demonstrate that we have a robust analysis pipeline that yields unbiased cosmological parameter inferences for the flagship 3x2pt DES Y1 analysis. We emphasize that the level of independent code development and subsequent code comparison as demonstrated in this paper is necessary to produce credible constraints from increasingly complex multi-probe analyses of current data.

79 ASTRONOMY AND ASTROPHYSICS↗

pnnl/apt

The current scripts utilize a well-known cluster analysis algorithm, called ordering points to identify the clustering structures (OPTICS), and an automatic cluster extraction algorithm for clustering analysis of 3D spatial data, specifically for the Atom Probe Tomography data. A hierarchical clustering scheme, together with density-based clustering, were implemented.

Wang, Jing↗

Alternatives to the 15% Rule

The third solicitation of the California Solar Initiative (CSI) Research, Development, Demonstration and Deployment (RD&D) Program established by the California Public Utility Commission (CPUC) is supporting the Electric Power Research Institute (EPRI), National Renewable Energy Laboratory (NREL), and Sandia National Laboratories (SNL) with collaboration from Pacific Gas and Electric (PG&E), Southern California Edison (SCE), and San Diego Gas and Electric (SDG&E), in research to improve the Utility Application Review and Approval process for interconnecting distributed energy resources to the distribution system. Currently this process is the most time - consuming of any step on the path to generating power on the distribution system. This CSI RD&D solicitation three project has completed the tasks of collecting data from the three utilities, clustering feeder characteristic data to attain representative feeders, detailed modeling of 16 representative feeders, analysis of PV impacts to those feeders, refinement of current screening processes, and validation of those suggested refinements. In this report each task is summarized to produce a final summary of all components of the overall project.

Broderick, Robert Joseph↗

OCEANIDS: Autonomous Data Acquisition, Management and Distribution System

OCEANIDS is a clearinghouse for mission essential and near-real-time satellite data streams. This viewgraph presentation describes this mission, and includes the following topics: 1) OCEANIDS Motivation; 2) High-Level Architecture; 3) OCEANIDS Features; 4) OCEANIDS GUI: Nodes; 5) OCEANIDS GUI: Cluster; 6) Data Streams; 7) Statistics; and 8) GHRSST-PP.

automated monitoring↗

Selection function of clusters in Dark Energy Survey year 3 data from cross-matching with South Pole Telescope detections

Context. Galaxy clusters selected based on overdensities of galaxies in photometric surveys provide the largest cluster samples. However, modeling the selection function of such samples is complicated by noncluster members projected along the line of sight (projection effects) and the potential detection of unvirialized objects (contamination). Aims. We empirically constrained the magnitude of these effects by cross-matching galaxy clusters selected in the Dark Energy Survey data with the redMaPPer algorithm with significant detections in three South Pole Telescope surveys (SZ, pol-ECS, pol-500d). Methods. For matched clusters, we augmented the redMaPPer catalog with the SPT detection significance. For unmatched objects we used the SPT detection threshold as an upper limit on the SZe signature. Using a Bayesian population model applied to the collected multiwavelength data, we explored various physically motivated models to describe the relationship between observed richness and halo mass. Results. Our analysis reveals a clear preference for models with an additional skewed scatter component associated with projection effects over a purely log-normal scatter model. We rule out significant contamination by unvirialized objects at the high-richness end of the sample. While dedicated simulations offer a well-fitting calibration of projection effects, our findings suggest the presence of redshift-dependent trends that these simulations may not have captured. Our findings highlight that modeling the selection function of optically detected clusters remains a complicated challenge that requires a combination of simulation and data-driven approaches.

79 ASTRONOMY AND ASTROPHYSICS↗

Signatures of Light Massive Relics on non-linear structure formation

ABSTRACT Cosmologies with Light Massive Relics (LiMRs) as a subdominant component of the dark sector are well-motivated from a particle physics perspective, and can also have implications for the σ8 tension between early and late time probes of clustering. The effects of LiMRs on the cosmic microwave background (CMB) and structure formation on large (linear) scales have been investigated extensively. In this paper, we initiate a systematic study of the effects of LiMRs on smaller, non-linear scales using cosmological N-body simulations; focusing on quantities relevant for photometric galaxy surveys. For most of our study, we use a particular model of non-thermal LiMRs but the methods developed generalizing to a large class of LiMR models – we explicitly demonstrate this by considering the Dodelson–Widrow velocity distribution. We find that, in general, the effects of LiMR on small scales are distinct from those of a ΛCDM universe, even when the value of σ8 is matched between the models. We show that weak lensing measurements around massive clusters, between ∼0.1 h−1Mpc and ∼10 h−1Mpc, should have sufficient signal-to-noise in future surveys to distinguish between ΛCDM and LiMR models that are tuned to fit both CMB data and linear scale clustering data at late times. Furthermore, we find that different LiMR cosmologies indistinguishable by conventional linear probes can be distinguished by non-linear probes if their velocity distributions are sufficiently different. LiMR models can, therefore, be best tested by jointly analyzing the CMB and late-time structure formation on both large and small scales.

79 ASTRONOMY AND ASTROPHYSICS↗

Visualizing Time-Varying Distribution Data in EOS Application

In this research, we have developed several novel visualization methods for spatial probability density function data. Our focus has been on 2D spatial datasets, where each pixel is a random variable, and has multiple samples which are the results of experiments on that random variable. We developed novel clustering algorithms as a means to reduce the information contained in these datasets; and investigated different ways of interpreting and clustering the data.

Shen, Han-Wei↗

A method of using cluster analysis to study statistical dependence in multivariate data

A technique is presented that uses both cluster analysis and a Monte Carlo significance test of clusters to discover associations between variables in multidimensional data. The method is applied to an example of a noisy function in three-dimensional space, to a sample from a mixture of three bivariate normal distributions, and to the well-known Fisher's Iris data.

Borucki, W. J.↗

Fusion and Fission Energy and Science Directorate and Information Technology Services Directorate HPC Cluster Reduction, Consolidation, and Savings in Data Center Space, Power, and Cooling

This report evaluates the benefits of decommissioning six legacy FFESD purchased HPC clusters and consolidating services and workloads into a new HPC cluster named HELIOS. The findings demonstrate significant reductions in the data center power and cooling requirements, data center footprint, and operational overhead, while simultaneously increasing computational capacity.

97 MATHEMATICS AND COMPUTING↗

A Complexity Agnostic Clustering Engine for Time Projection Chambers and its Implementation in FPGA

A clustering functional block implemented in field-programable-gate-array (FPGA) for time projection chambers (TPC) operating with predictable time regardless the complexity of the event is described in this paper. The clustering functional block reorganizes input data and the hits data belonging to the same clusters are output together for further process in the later stages. The clustering operation consists of two phases, data filling phase and data outputting phase, and the later uses the same number of clock cycles as the data filling phase. The clustering block can accommodate events with arbitrary number of clusters and number of hits per cluster as long as the total number of hits is within a predesigned limit. The operation time is exactly twice of the data filling time with no residual O(n2) term. The clustering block has been implemented with operating frequency of 200 MHz in a low-cost FPGA evaluation module and test results confirm the expected performance.

Wu, Jinyuan [Fermilab] (ORCID:0000000344329521)↗

M-DAS: System for multispectral data analysis

M-DAS is a ground data processing system designed for analysis of multispectral data. M-DAS operates on multispectral data from LANDSAT, S-192, M2S and other sources in CCT form. Interactive training by operator-investigators using a variable cursor on a color display was used to derive optimum processing coefficients and data on cluster separability. An advanced multivariate normal-maximum likelihood processing algorithm was used to produce output in various formats: color-coded film images, geometrically corrected map overlays, moving displays of scene sections, coverage tabulations and categorized CCTs. The analysis procedure for M-DAS involves three phases: (1) screening and training, (2) analysis of training data to compute performance predictions and processing coefficients, and (3) processing of multichannel input data into categorized results. Typical M-DAS applications involve iteration between each of these phases. A series of photographs of the M-DAS display are used to illustrate M-DAS operation.

Johnson, R. H.↗