Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Data Analytics Applied to Coal Fired Boilers for Detecting Leaks

Data analytics were used to detect boiler leaks from five different coal-fired boilers including both subcritical and supercritical systems. Discriminant functions were developed that detected leaks up to two weeks prior to forced plant shutdowns for repairs. The leaks were identified to occur at different sections of the boiler for each plant, including waterwalls, economizer and superheater using conventional process measurement data. Leaking conditions were detected with a high degree of confidence (≪ 1% misclassified observations) and were able to distinguish normal operations from those time periods with steam leaks even while operating the power plants in power cycling mode.Multivariable statistical analyses, including Principal Component (PCA), cluster, and Fischer Discriminant Analysis (FDA) were used to characterize the leak occurrence. Normal and operational states with steam leaks were provided in the original process datasets. These datasets were split into two different groups for training and validation purposes. The data were sorted chronologically, and every third observation was assigned to training the Discriminant Function Model (DFM) while the rest were reserved for validation. PCA was used to reduce dimensionality of the original datasets. Canonical and FDA analyses were used to investigate the relationship between process variables. The outcome of the analyses revealed that nearly 35,000 observations were classified correctly; less than 0.05% of total observations were misclassified to be leaking, i.e. both false positives and false negatives.

Indrawan, Natarianto↗

Stability of Cool Cores during Galaxy Cluster Growth: A Joint Chandra/SPT Analysis of 67 Galaxy Clusters along a Common Evolutionary Track Spanning 9 Gyr

We present the results of a joint analysis of Chandra X-ray and South Pole Telescope (SPT) Sunyaev–Zel’dovich (SZ) observations targeting the first sample of galaxy clusters at 0.3 < z < 1.3, selected to be the progenitors of well-studied nearby clusters based on their expected accretion rate. We develop a new procedure in order to tackle the analysis challenge that is estimating the intracluster medium (ICM) properties of low-mass and high-redshift clusters with ~150 X-ray counts. One of the dominant sources of uncertainty on the ICM density profile estimated with a standard X-ray analysis with such shallow X-ray data is due to the systematic uncertainty associated with the ICM temperature obtained through the analysis of the background-dominated X-ray spectrum. We show that we can decrease the uncertainty on the density profile by a factor varying between 2 and 8 with a joint deprojection of the X-ray surface brightness profile measured by Chandra and the SZ-integrated Compton parameter available in the SPT cluster catalog. We apply this technique to the whole sample of 67 clusters in order to track the evolution of the ICM core density during cluster growth. We confirm that the evolution of the gas density profile is well modeled by the combination of a fixed core and a self-similarly evolving non-cool-core profile. We show that the fraction of cool cores in this sample is remarkably stable with redshift although clusters have gained a factor of ~4 in total mass over the past ~9 Gyr.

79 ASTRONOMY AND ASTROPHYSICS↗

Enantioselective catalysts based on metal-organic framework-supported nucleotides

Adenosine triphosphate (ATP) and other nucleotides can be irreversibly bound to the metal-organic framework (MOF) MIL-101(Cr). Analysis of X-ray diffraction data suggests that the location of the adsorbed ATP molecule is in proximity of the Cr 3 clusters. Solid-state NMR and DFT calculations indicate that ATP is bound to MIL-101(Cr) through linkages of the terminal phosphate group with Cr(III) of the framework. In the presence of Cu(II) ions, the MOF-supported nucleotides can function as stable and reusable enantioselective heterogeneous catalysts for reactions like Diels-Alder and Michael addition. In conclusion, compared to the corresponding homogeneous nucleotide-based artificial metalloenzymes (ArMs), the MOF-supported nucleotide-based ArMs exhibit significantly enhanced activity and selectivity in certain cases, demonstrating their potential as a new class of enantioselective heterogeneous catalysts.

36 MATERIALS SCIENCE↗

Electronic Spin Relaxation and Clustering in High-Pressure High-Temperature Synthesized Microcrystalline Diamond Particles with Reduced Nitrogen Content

The negatively charged nitrogen-vacancy (NV – ) color center in diamonds is widely studied because of numerous applications of this unique quantum system in sensing and quantum information sciences. While substitutional nitrogen is required to form the NV – centers in diamond, it also yields other paramagnetic defects─primarily the neutrally charged substitutional nitrogen centers (P1)─that decrease NV – spin coherence, which in turn degrades performance in applications. Herein, we investigate high-pressure high-temperature synthesized diamond microparticles (ca. 140–185 μm) having lower─ranging from 3 to 38 ppm─than the typical nitrogen content of type 1b diamond ( ca. 100 ppm and higher) typically used for the production of fluorescent diamond particles with NV – centers. A suite of electron paramagnetic resonance, optically detected magnetic resonance, and nuclear magnetic resonance methods are used to characterize spin properties of P1 and NV – centers in the particles. Upon decreasing the nitrogen content from 29 to 3 ppm, the ensemble NV – T 2 relaxation time increased by about 3-fold as measured directly in the Hahn Echo experiment at magnetic field of 1.2 T. Analysis of electronic relaxation of P1 centers revealed the existence of at least two distinct populations of P1 centers, consisting of fast and slower relaxing spins and allowed for an estimation of local concentrations. Even with <10 ppm nitrogen contents, the analysis indicated a highly heterogeneous distribution of P1 centers, suggesting the possibility of P1 spin clustering even at low nitrogen concentrations. The combined data demonstrate that the particles prepared from HPHT diamond with a low nitrogen content offer improved spin properties that are beneficial for NV – sensing applications.

Carbon↗

Transferable Water Potentials Using Equivariant Neural Networks

Machine learning interatomic potentials (MLIPs) have emerged as a technique that promises quantum theory accuracy for reduced cost. It has been proposed [J. Chem. Phys. 2023, 158, 084111] that MLIPs trained on solely liquid water data cannot accurately transfer to the vapor–liquid equilibrium while recovering the many-body decomposition (MBD) analysis of gas-phase water clusters. This suggests that MLIPs do not directly learn the physically correct interactions of water molecules, limiting transferability. In this work, we show that MLIPs using equivariant architecture and trained on 3200 liquid water structures reproduces liquid-phase water properties (e.g., density within 0.003 g/cm 3 between 230 and 365 K), vapor–liquid equilibrium properties up to 550 K, the MBD analysis of gas-phase water cluster up to six-body interactions, and the relative energy and the vibrational density of states of ice phases. We show that potentials developed using equivariant MLIPs allow transferability for arbitrary phases of water that remain stable in nanosecond long simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Applying Magnetic Curvature to MMS Data to Identify Thin Current Sheets Relative to Tail Reconnection

Thin current sheets (TCSs) have been postulated to be a necessary precondition for reconnection onset. Magnetic reconnection X-lines in the magnetotail have been observed to be more common duskward of midnight. We take advantage of the MMS tetrahedral formation during the 2017–2020 MMS tail seasons to calculate the thickness of the cross-tail neutral sheet relative to ion gyroradius. While a similar technique was applied to Cluster data, current sheet thickness over a broader range of radial distances has not been robustly explored before this study. We compare our analysis to recent theories regarding mechanisms of tail current sheet thinning and to recent simulations. We find MMS spent more than twice as long in ion-scale TCSs in the pre-midnight sector than post-midnight, despite nearly even plasma sheet dwell time. The dawn-dusk asymmetry in the distribution of Ion Diffusion Regions, as previously reported in relation to regions of TCSs, is also analyzed.

79 ASTRONOMY AND ASTROPHYSICS↗

Informed total-error-minimizing priors: Interpretable cosmological parameter constraints despite complex nuisance effects

While Bayesian inference techniques are standard in cosmological analyses, it is common to interpret resulting parameter constraints with a frequentist intuition. This intuition can fail, for example, when marginalizing high-dimensional parameter spaces onto subsets of parameters, because of what has come to be known as projection effects or prior volume effects. We present the method of informed total-error-minimizing (ITEM) priors to address this problem. An ITEM prior is a prior distribution on a set of nuisance parameters, such as those describing astrophysical or calibration systematics, intended to enforce the validity of a frequentist interpretation of the posterior constraints derived for a set of target parameters (e.g., cosmological parameters). Our method works as follows. For a set of plausible nuisance realizations, we generate target parameter posteriors using several different candidate priors for the nuisance parameters. We reject candidate priors that do not accomplish the minimum requirements of bias (of point estimates) and coverage (of confidence regions among a set of noisy realizations of the data) for the target parameters on one or more of the plausible nuisance realizations. Of the priors that survive this cut, we select the ITEM prior as the one that minimizes the total error of the marginalized posteriors of the target parameters. As a proof of concept, we applied our method to the density split statistics measured in Dark Energy Survey Year 1 data. We demonstrate that the ITEM priors substantially reduce prior volume effects that otherwise arise and that they allow for sharpened yet robust constraints on the parameters of interest.

79 ASTRONOMY AND ASTROPHYSICS↗

A full year of aerosol size distribution data from the central Arctic under an extreme positive Arctic Oscillation: insights from the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition

Abstract. The Arctic environment is rapidly changing due to accelerated warming in the region. The warming trend is driving a decline in sea ice extent, which thereby enhances feedback loops in the surface energy budget in the Arctic. Arctic aerosols play an important role in the radiative balance and hence the climate response in the region, yet direct observations of aerosols over the Arctic Ocean are limited. In this study, we investigate the annual cycle in the aerosol particle number size distribution (PNSD), particle number concentration (PNC), and black carbon (BC) mass concentration in the central Arctic during the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition. This is the first continuous, year-long data set of aerosol PNSD ever collected over the sea ice in the central Arctic Ocean. We use a k-means cluster analysis, FLEXPART simulations, and inverse modeling to evaluate seasonal patterns and the influence of different source regions on the Arctic aerosol population. Furthermore, we compare the aerosol observations to land-based sites across the Arctic, using both long-term measurements and observations during the year of the MOSAiC expedition (2019–2020), to investigate interannual variability and to give context to the aerosol characteristics from within the central Arctic. Our analysis identifies that, overall, the central Arctic exhibits typical seasonal patterns of aerosols, including anthropogenic influence from Arctic haze in winter and secondary aerosol processes in summer. The seasonal pattern corresponds to the global radiation, surface air temperature, and timing of sea ice melting/freezing, which drive changes in transport patterns and secondary aerosol processes. In winter, the Norilsk region in Russia/Siberia was the dominant source of Arctic haze signals in the PNSD and BC observations, which contributed to higher accumulation-mode PNC and BC mass concentrations in the central Arctic than at land-based observatories. We also show that the wintertime Arctic Oscillation (AO) phenomenon, which was reported to achieve a record-breaking positive phase during January–March 2020, explains the unusual timing and magnitude of Arctic haze across the Arctic region compared to longer-term observations. In summer, the aerosol PNCs of the nucleation and Aitken modes are enhanced; however, concentrations were notably lower in the central Arctic over the ice pack than at land-based sites further south. The analysis presented herein provides a current snapshot of Arctic aerosol processes in an environment that is characterized by rapid changes, which will be crucial for improving climate model predictions, understanding linkages between different environmental processes, and investigating the impacts of climate change in future Arctic aerosol studies.

54 ENVIRONMENTAL SCIENCES↗

Tomographic galaxy clustering with the Subaru Hyper Suprime-Cam first year public data release

We analyze the clustering of galaxies in the first public data release of the HSC Subaru Strategic Program. Despite the relatively small footprints of the observed fields, the data are an excellent proxy for the deep photometric datasets that will be acquired by LSST, and are therefore an ideal test bed for the analysis methods being implemented by the LSST DESC. We select a magnitude limited sample with i<24.5 and analyze it in four redshift bins covering 0.15≲z≲1.5. We carry out a Fourier-space analysis of the two-point clustering of this sample, including all auto- and cross-correlations. We demonstrate the use of map-level deprojection methods to account for fluctuations in the galaxy number density caused by observational systematics. Through an HOD analysis, we place constraints on the characteristic halo masses of this sample, finding a good fit up to scales k max = 1Mpc –1 , including both auto- and cross-correlations. Our results show monotonically decreasing average halo masses, which can be interpreted in terms of the drop-out of red galaxies at high redshifts for a flux-limited sample. In terms of photometric redshift systematics, we show that additional care is needed in order to marginalize over uncertainties in the redshift distribution in galaxy clustering, and that these uncertainties can be constrained by including cross-correlations. We are able to make a ~3σ detection of lensing magnification in the HSC data. Our results are stable to variations in σ 8 and Ω c and we find constraints that agree well with measurements from Planck and low-redshift probes. Lastly, we use our pipeline to study the clustering of galaxies as a function of limiting flux, and provide a simple fitting function for the linear galaxy bias for magnitude limited samples as a function of limiting magnitude and redshift.

79 ASTRONOMY AND ASTROPHYSICS↗

𝑅-matrix analysis of 22 Ne states populated in 18 O ⁡(𝛼,𝛼) resonant elastic scattering

The properties of resonances in the 18 O + 𝛼 interaction are important for analyses of astrophysical processes and for understanding of the development of the 𝛼-cluster structure with addition of neutrons to 4N nuclei. Previous high-energy resolution measurements have not been treated using the contemporary 𝑅-matrix approach. We have performed an 𝑅-matrix analysis of the resonant data [D. Powers, J. K. Bair, J. L. C. Ford, Jr., and H. B. Willard, Phys. Rev. 134, B1237 (1964)] and obtained new data on 15 levels in the excitation region of 11.7–12.5 MeV 22 Ne. It appears that the model calculations did not predict the observed tendency in the 20–22 Ne 𝛼 clustering. This paper is the first step of our program of investigation of the 22 Ne spectrum in resonant reactions.

6 ≤ A ≤ 19↗

Analysis of Interpretable Data Representations for 4D-STEM Using Unsupervised Learning

Abstract Understanding the structure of materials is crucial for engineering devices and materials with enhanced performance. Four-dimensional scanning transmission electron microscopy (4D-STEM) is capable of mapping nanometer-scale local crystallographic structure over micron-scale field of views. However, 4D-STEM datasets can contain tens of thousands of images from a wide variety of material structures, making it difficult to automate detection and classification of structures. Traditional automated analysis pipelines for 4D-STEM focus on supervised approaches, which require prior knowledge of the material structure and cannot describe anomalous or deviant structures. In this article, a pipeline for engineering 4D-STEM feature representations for unsupervised clustering using non-negative matrix factorization (NMF) is introduced. Each feature is evaluated using NMF and results are presented for both simulated and experimental data. It is shown that some data representations more reliably identify overlapping grains. Additionally, real space refinement is applied to identify spatially distinct sample regions, allowing for size and shape analysis to be performed. This work lays the foundation for improved analysis of nanoscale structural features in materials that deviate from expected crystallographic arrangement using 4D-STEM.

Bruefach, Alexandra (ORCID:0000000209323477)↗

Spatio-temporal multivariate cluster evolution analysis for detecting and tracking climate impacts

Recent years have seen a growing concern about climate change and its impacts. While Earth System Models (ESMs) can be invaluable tools for studying the impacts of climate change, the complex coupling processes encoded in ESMs and the large amounts of data produced by these models, together with the high internal variability of the Earth system, can obscure important source-to-impact relationships. Here, this paper presents a novel and efficient unsupervised data-driven approach for detecting statistically-significant impacts and tracing spatio-temporal source-impact pathways in the climate through a unique combination of ideas from anomaly detection, clustering and Natural Language Processing (NLP). Using as an exemplar the 1991 eruption of Mount Pinatubo in the Philippines, we demonstrate that the proposed approach is capable of detecting known post-eruption impacts/events. We additionally describe a methodology for extracting meaningful sequences of post-eruption impacts/events by using NLP to efficiently mine frequent multivariate cluster evolutions, which can be used to confirm or discover the chain of physical processes between a climate source and its impact(s).

Anomaly detection↗

Geometry, spin coupling, and dielectric control of redox potentials in [4Fe–4S] Clusters

Iron–sulfur (Fe–S) clusters are common biological cofactors that facilitate vital redox reactions. Despite extensive research, the molecular basis of redox potential tuning in ferredoxin-like proteins remains an active area of debate. In this study, we combine statistical analysis of over one thousand [4Fe–4S]-containing protein structures from the Protein Data Bank (PDB) with broken-symmetry and extended broken-symmetry density functional theory to examine how cysteine ligand orientations and environmental screening affect redox properties of the clusters. We identified five main ligand configurations, three of which are predominant in natural structures. Among these, the adiabatic electron affinity differs by less than 0.1 V, indicating that, while geometry plays a secondary role, it allows localized fine-tuning of redox properties. In contrast, electrostatic and solvation effects primarily determine the overall potential range.

Computational Chemistry↗

Science Validation for Dark Energy Research with Optical Imaging Surveys

The Universe has been expanding at an accelerating rate over the past several billion years, as though an unknown form of dark energy permeates all of space. The ultimate scientific goal of the proposed research is to distinguish between different physical mechanisms that could account for this observed accelerated expansion, for example, the zero-point energy of the vacuum, a dynamical form of energy that varies in time and/or space, or a modification to our theory of gravity. Wide- field optical imaging surveys of the night sky can test these competing models by measuring both the cosmic expansion history and the growth of large-scale structure. To this end, the Dark Energy Survey (DES) has cataloged several hundred million galaxies and thousands of supernovae. The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), will enlarge the census to billions of galaxies and hundreds of thousands of supernovae. A critical question for these dark energy experiments is whether systematic uncertainties can continue to be controlled at a level to keep pace with the statistical precision offered by such enormous datasets. The immediate research objectives of this project were (1) to prepare and validate input datasets that are the foundation of cosmological analyses with DES, and (2) to prepare for value-added characterization of Rubin Observatory commissioning data to inform early operations and accelerate the realization of dark energy science from LSST data products. For DES, we assembled and curated cosmology-ready data releases that include value-added components such as enhanced photometric and astrometric calibrations, alternative source extraction algorithms, maps of the survey coverage and survey conditions, object classifications, object quality selections, galaxy shapes, and photometric redshifts. We used the galaxy clustering technique to validate the photometric redshift distributions of various galaxy samples to be used as lenses in combined studies of galaxy clustering and weak gravitational lensing. The galaxy clustering redshift analysis was enhanced by use of a larger sample of reference galaxies from the eBOSS spectroscopic survey that extends to higher redshifts. For LSST, we prepared for science validation studies of commissioning data aimed at dark energy science capability that extend beyond the normative system-level tests to be done by the Rubin Observatory Construction Project. We identified a set of proposed survey strategies and candidate target fields that could be observed during the commissioning period to enhance science validation activities related to studies of dark energy.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

A Comprehensive Analysis of NGC 2158 in the Gaia Era: Photometric Parameters, Apex, and Orbit

We present an investigation of NGC 2158 using Gaia DR2 data. We identified 800 most likely cluster members with membership probability higher than 90%. The mean proper motions of this object are determined as (μ {sub x} = −0.203 ± 0.003, μ {sub y} = −1.99 ± 0.004) mas yr{sup −1}. The limiting radius, log(age), and distance of the cluster are obtained as 23.5 arcmin, 9.38 ± 0.04 Gyr, and 4.69 ± 0.22 kpc, respectively. The overall mass function slope (0.93 ± 0.14) is flatter than the Salpeter value (1.35) within the mass range of 1.17–1.44 M {sub ⊙}. This cluster also shows the mass-segregation effect and our study demonstrates that NGC 2158 is a dynamically relaxed open cluster. Using the apex diagram (AD), the apex coordinates of the cluster are obtained in different ways and are examined using the (μ {sub U},μ {sub T}) diagram. The best value of the apex coordinates is determined as A = 8724 ± 160, D = −3661 ± 530. We also determined the orbit of the cluster and found that NGC 2158 moves almost in the solar antapex direction. The resulting spatial velocity of NGC 2158 is 51 km s{sup −1}. A significant oscillation along the Z-coordinate up to 529 pc is detected. Various scenarios regarding the origin of this cluster are also discussed.

42 ENGINEERING↗

Dark Energy Survey Year 3 results: Simulation-based cosmological inference with wavelet harmonics, scattering transforms, and moments of weak lensing mass maps. Validation on simulations

Beyond-two-point statistics contain additional information on cosmological as well as astrophysical and observational (systematics) parameters. In this methodology paper we provide an end-to-end simulationbased analysis of a set of Gaussian and non-Gaussian weak lensing statistics using detailed mock catalogs of the Dark Energy Survey (DES). Here, we implement: 1) second and third moments; 2) wavelet phase harmonics (WPH); 3) the scattering transform (ST). Our analysis is fully based on simulations, it spans a space of seven $νw$CDM cosmological parameters, and it forward models the most relevant sources of systematics of the data (masks, noise variations, clustering of the sources, intrinsic alignments, and shear and redshift calibration). We implement a neural network compression of the summary statistics, and we estimate the parameter posteriors using a likelihood-free-inference approach. We validate the pipeline extensively, and we find that WPH exhibits the strongest performance when combined with second moments, followed by ST, and then by third moments. The combination of all the different statistics further enhances constraints with respect to second moments, up to 25 percent, 15 percent, and 90 percent for S 8 , Ω m , and the figure-of-merit FoM S8;Ωm , respectively. We further find that non-Gaussian statistics improve constraints on w and on the amplitude of intrinsic alignment with respect to second moments constraints. The methodological advances presented here are suitable for application to Stage IV surveys from Euclid, Rubin-LSST, and Roman with additional validation on mock catalogs for each survey. In a companion paper we present an application to DES Year 3 data.

79 ASTRONOMY AND ASTROPHYSICS↗

Analysis of Completion Design Impact on Cluster Efficiency and Pressure-Based Well Communication in HFTS-2 Delaware Basin

The Hydraulic Fracturing Test Site 2 (HFTS-2) is a joint industry project in the Delaware basin to advance hydraulic fracturing understanding and improve productivity in shale reservoirs. The project integrates multi-disciplinary approaches to evaluate different completion designs, well spacing, inter-well communication, and stimulated rock volume, among other factors. This paper focused on two major areas related to hydraulic fracture performance. First, an analysis of different completion designs on cluster efficiency based on near well Distributed Acoustic Sensing (DAS). Second, an evaluation of well and completion designs on inter-well fracture driven interactions (FDIs) based on downhole pressure monitoring. Fluid/sand distribution and cluster efficiency analyses were based on near wellbore DAS data collected from two adjacent horizontal wells completed in two different landing zones in the Wolfcamp formation. These wells had different completion designs aiming to evaluate the effect of normal vs. extended stage lengths, perforation hole tapering and limited entry. Standard deviation from ideal fluid/sand distribution and waterfall plots were used to evaluate cluster efficiency for each design and stage. Inter-well FDIs analysis was conducted among the horizontal wells and a vertical monitor well. One horizontal well served as the monitor well while the other horizontal well was being treated. The vertical well was instrumented with downhole pressure and temperature gauges to aid monitoring fracture height growth. The pressure response during and after fracturing was characterized based on maximum pressure increase value and slope. Pressure response vs. FDIs trigger factors such distance, cluster efficiency and stage fluid volume were also analyzed. Based on the different completion and perforation designs tested, DAS analysis suggests that limited entry design worked best. Extreme limited entry showed the potential of high perforation erosion and reduced cluster efficiency. Here, the limited entry and tapered perforation design demonstrated potential to improve the cluster efficiency for extended stage lengths. Pressure monitoring across formation units proved to be critical to understand fracture interactions and fracture vertical growth. Pressure communication across different formation units during hydraulic fracturing operation indicate fractures grew upwards during Wolfcamp wells fracturing. However, this pressure communication dissipated over time. High intensity FDIs were recorded when the frac stages Downloaded from http://onepetro.org/URTECONF/proceedings-pdf/21URTC/1-21URTC/D011S018R001/2477607/urtec-2021-5289-ms.pdf/1 by Carol Worster on 28 February 2022 URTeC 5289 were closer to the pressure gauge location in the monitor wells. Some of these stages that produced high intensity FDIs also had high fluid volume per cluster and low cluster efficiency. The multi-disciplinary and high-quality data collected from HFTS-2 helped to further understand why completion approaches such as limited entry and tapered perforation design are successful in improving cluster efficiency. The DAS data combined with downhole high-resolution pressure measurements also helped to quantify the effect of lower cluster efficiency data on the incidence and intensity of FDIs.

58 GEOSCIENCES↗

Efficient Clustering of Software Vulnerabilities using Self Organizing Map (SOM)

The common vulnerabilities and exposures (CVE) database was created with a mission to ``identify, define, and catalog publicly disclosed cybersecurity vulnerabilities''. This rich body of information can be used to enable rapid and efficient response to secure and defend cyber operations and protect critical cyber infrastructure. The main goal of this paper is to develop a visual analytics tool to enable deep analysis of CVEs using unsupervised clustering techniques. We enhance our analysis by first mapping CVEs to hierarchical-classes in Common Weakness Enumeration (CWE) using information in the National Vulnerability Database (NVD). Both the mapping and the numerical representation of CVEs are enabled by V2W-BERT, which uses natural language processing of the extensive information in NVD to generate a large tabular database of 137,226 CVE entries from 1999 to 2020, where each CVE is represented by a vector of 768 numerical features. The vectorized data is processed by Self-Organizing Maps (SOM), which is an unsupervised machine learning technique for dimensionality reduction, visual representation and clustering. Using a Torus map of 6417 units, we achieve ~10-fold data compression of ~140k CVEs using SOM. The trained map is further clustered using standard K-means clustering into 138 clusters of CVEs. We conducted a brief investigation of the rich mapping of CVEs to best-matching-units to K-means clusters, as well as CVEs to CWEs. For example, this novel mapping provided insight into the role of CWE-59 and CWE-264 in several CVEs that is otherwise hard to explore in the original data. We conclude that our this novel approach will not only enable deep analysis of the complex relationships between CVEs and CWEs, but also a mechanism to quickly respond to and design mitigation actions for rapidly evolving vulnerabilities that have not been mapped to existing CWEs.

Panchal, Khyati↗