Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A framework to measure the properties of intergalactic metal systems with two-point flux statistics

ABSTRACT The abundance, temperature, and clustering of metals in the intergalactic medium are important parameters for understanding their cosmic evolution and quantifying their impact on cosmological analysis with the Ly α forest. The properties of these systems are typically measured from individual quasar spectra redward of the quasar’s Ly α emission line, yet that approach may provide biased results due to selection effects. We present an alternative approach to measure these properties in an unbiased manner with the two-point statistics commonly employed to quantify large-scale structure. Our model treats the observed flux of a large sample of quasar spectra as a continuous field and describes the one-dimensional, two-point statistics of this field with three parameters per ion: the abundance (column density distribution), temperature (Doppler parameter), and clustering (cloud–cloud correlation function). We demonstrate this approach on multiple ions (e.g. ${\rm C\, \small {\rm IV}}$ , ${\rm Si\, \small {\rm IV}}$ , and ${\rm Mg\, \small {\rm II}}$ ) with early data from the Dark Energy Spectroscopic Instrument (DESI) and high-resolution spectra from the literature. Our initial results show some evidence that the ${\rm C\, \small {\rm IV}}$ abundance is higher than previous measurements and evidence for abundance evolution over time. The first full year of DESI observations will have over an order of magnitude more quasar spectra than this study. In a future paper, we will use those data to measure the growth of clustering and its impact on the Ly α forest, as well as test other DESI analysis infrastructure such as the pipeline noise estimates and the resolution matrix.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2010

Reducing the Volume of NASA Earth-Science Data

A computer program reduces data generated by NASA Earth-science missions into representative clusters characterized by centroids and membership information, thereby reducing the large volume of data to a level more amenable to analysis. The program effects an autonomous data-reduction/clustering process to produce a representative distribution and joint relationships of the data, without assuming a specific type of distribution and relationship and without resorting to domain-specific knowledge about the data. The program implements a combination of a data-reduction algorithm known as the entropy-constrained vector quantization (ECVQ) and an optimization algorithm known as the differential evolution (DE). The combination of algorithms generates the Pareto front of clustering solutions that presents the compromise between the quality of the reduced data and the degree of reduction. Similar prior data-reduction computer programs utilize only a clustering algorithm, the parameters of which are tuned manually by users. In the present program, autonomous optimization of the parameters by means of the DE supplants the manual tuning of the parameters. Thus, the program determines the best set of clustering solutions without human intervention.

Lee, Seungwon↗
NASA NTRS1983

An interactive Barnes objective map analysis scheme for use with satellite and conventional data

The Barnes (1973) objective map analysis scheme is employed to develop an interactive analysis package for assessing the impact of satellite-derived data on analyses of conventional meteorological data sets. The method permits modification of the values of input parameters in the objective analysis within objectively determined, internally set limits. The effects of the manipulations are rapidly displayed, and methods are included for assimilating the spatially clustered characteristics of satellite data and the various horizontal resolutions of the data types. Data sets from the SESAME rawinsonde wind data with uniform spatial distribution, with the same data set plus satellite cloud motion data, and a data set from the atmospheric sounder radiometer on the GOES satellite were analyzed as examples. The scheme is demonstrated to recover details after two iterations through the data.

Koch, S. E.↗
NASA NTRS1997

Analysis of Crystallization Kinetics

A realistic computer model for polymorphic crystallization (i.e., initial and final phases with identical compositions), which includes time-dependent nucleation and cluster-size-dependent growth rates, is developed and tested by fits to experimental data. Model calculations are used to assess the validity of two of the more common approaches for the analysis of crystallization data. The effects of particle size on transformation kinetics, important for the crystallization of many systems of limited dimension including thin films, fine powders, and nanoparticles, are examined.

Kelton, Kenneth F.↗
DOE OSTI2023

Photometry and astrometry with JWST – III. A NIRCam-Gaia DR3 analysis of the open cluster NGC 2506

ABSTRACT In the third paper of this series aimed at developing the tools for analysing resolved stellar populations using the cameras on board of the James Webb Space Telescope (JWST), we present a detailed multiband study of the 2 Gyr Galactic open cluster NGC 2506. We employ public calibration data sets collected in multiple filters to: (i) derive improved effective Point Spread Functions (ePSFs) for 10 NIRCam filters; (ii) extract high-precision photometry and astrometry for stars in the cluster, approaching the main sequence (MS) lower mass of ∼0.1 M⊙; and (iii) take advantage of the synergy between JWST and Gaia DR3 to perform a comprehensive analysis of the cluster’s global and local properties. We derived a MS binary fraction of ∼57.5 per cent, extending the Gaia limit (∼0.8 M⊙) to lower masses (∼0.4 M⊙) with JWST. We conducted a study on the mass functions (MFs) of NGC 2506, mapping the mass segregation with Gaia data, and extending MFs to lower masses with the JWST field. We also combined information on the derived MFs to infer an estimate of the cluster present-day total mass. Lastly, we investigated the presence of white dwarfs (WDs) and identified a strong candidate. However, to firmly establish its cluster membership, as well as that of four other WD candidates and of the majority of faint low-mass MS stars, further JWST equally deep observations will be required. We make publicly available catalogues, atlases, and the improved ePSFs.

Nardiello, D. (ORCID:0000000311493659)↗
DOE OSTI2020

Rapid Identification of X-ray Diffraction Patterns Based on Very Limited Data by Interpretable Convolutional Neural Networks

Large volumes of data from material characterizations call for rapid and automatic data analysis to accelerate materials discovery. Herein, we report a convolutional neural network (CNN) that was trained based on theoretical data and very limited experimental data for fast identification of experimental X-ray diffraction (XRD) patterns of metal–organic frameworks (MOFs). To augment the data for training the model, noise was extracted from experimental data and shuffled; then it was merged with the main peaks that were extracted from theoretical spectra to synthesize new spectra. For the first time, one-to-one material identification was achieved. Theoretical MOFs patterns (1012) were augmented to a whole data set of 72 864 samples. It was then randomly shuffled and split into training (58 292 samples) and validation (14 572 samples) data sets at a ratio of 4:1. For the task of discriminating, the optimized model showed the highest identification accuracy of 96.7% for the top 5 ranking on a test data set of 30 hold-out samples. Neighborhood component analysis (NCA) on the experimental XRD samples shows that the samples from the same material are clustered in groups in the NCA map. Analysis on the class activation maps of the last CNN layer further discloses the mechanism by which the CNN model successfully identifies individual MOFs from the XRD patterns. Furthermore, this CNN model trained by the data augmentation technique would not only open numerous potential applications for identifying XRD patterns for different materials, but also pave avenues to autonomously analyze data by other characterization tools such as FTIR, Raman, and NMR spectroscopies.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Implementation Aspects of Smart Grids Cyber-Security Cross-Layered Framework for Critical Infrastructure Operation

Communication networks in power systems are a major part of the smart grid paradigm. It enables and facilitates the automation of power grid operation as well as self-healing in contingencies. Such dependencies on communication networks, though, create a roam for cyber-threats. An adversary can launch an attack on the communication network, which in turn reflects on power grid operation. Attacks could be in the form of false data injection into system measurements, flooding the communication channels with unnecessary data, or intercepting messages. Using machine learning-based processing on data gathered from communication networks and the power grid is a promising solution for detecting cyber threats. In this paper, a co-simulation of cyber-security for cross-layer strategy is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection as well as identification of anomalies in the operation of the power grid. The framework is implemented on the IEEE 118-bus system. The system is constructed in Mininet to simulate a communication network and obtain data for analysis. A distributed three controller software-defined networking (SDN) framework is proposed that utilizes the Open Network Operating System (ONOS) cluster. According to the findings of our suggested architecture, it outperforms a single SDN controller framework by a factor of more than ten times the throughput. This provides for a higher flow of data throughout the network while decreasing congestion caused by a single controller’s processing restrictions. Furthermore, our CECD-AS approach outperforms state-of-the-art physics and machine learning-based techniques in terms of attack classification. The performance of the framework is investigated under various types of communication attacks.

cross-layered↗
DOE OSTI2025

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗
NASA NTRS1994

Multivariate statistical analysis software technologies for astrophysical research involving large data bases

We developed a package to process and analyze the data from the digital version of the Second Palomar Sky Survey. This system, called SKICAT, incorporates the latest in machine learning and expert systems software technology, in order to classify the detected objects objectively and uniformly, and facilitate handling of the enormous data sets from digital sky surveys and other sources. The system provides a powerful, integrated environment for the manipulation and scientific investigation of catalogs from virtually any source. It serves three principal functions: image catalog construction, catalog management, and catalog analysis. Through use of the GID3* Decision Tree artificial induction software, SKICAT automates the process of classifying objects within CCD and digitized plate images. To exploit these catalogs, the system also provides tools to merge them into a large, complete database which may be easily queried and modified when new data or better methods of calibrating or classifying become available. The most innovative feature of SKICAT is the facility it provides to experiment with and apply the latest in machine learning technology to the tasks of catalog construction and analysis. SKICAT provides a unique environment for implementing these tools for any number of future scientific purposes. Initial scientific verification and performance tests have been made using galaxy counts and measurements of galaxy clustering from small subsets of the survey data, and a search for very high redshift quasars. All of the tests were successful, and produced new and interesting scientific results. Attachments to this report give detailed accounts of the technical aspects for multivariate statistical analysis of small and moderate-size data sets, called STATPROG. The package was tested extensively on a number of real scientific applications, and has produced real, published results.

Djorgovski, S. George↗
NASA NTRS1986

Solar microwave millisecond spike at 2.84 GHz

Using the high time resolution of 1 ms, the data of solar microwave millisecond spike (MMS) event was recorded more than two hundred times at the frequency of 2.84 GHz at Beijing (Peking) Observatory since May 1981. A preliminary analysis was made. It can be seen from the data that the MMS-events have a variety of the fast activities such as the dispersed and isolated spikes, the clusters of the crowded spikes, the weak spikes superimposed on the noise background, and the phenomena of absorption. The marked differences from that observed with lower time resolution are presented. Using the data, a valuable statistical analysis was made. There are close correlations between MMS-events and hard X-ray bursts, and fast drifting bursts. The MMS events are highly dependent on the type of active regions and the magnetic field configuration. It seems to be crucial to find out the accurate positions on the active region where the MMS-events happen and to make co-operative observations at different bands during the special period when specific active regions appear on the solar disk.

Fu, Qi-Jun↗
NASA NTRS2011

CFD Based Computations of Flexible Helicopter Blades for Stability Analysis

As a collaborative effort among government aerospace research laboratories an advanced version of a widely used computational fluid dynamics code, OVERFLOW, was recently released. This latest version includes additions to model flexible rotating multiple blades. In this paper, the OVERFLOW code is applied to improve the accuracy of airload computations from the linear lifting line theory that uses displacements from beam model. Data transfers required at every revolution are managed through a Unix based script that runs jobs on large super-cluster computers. Results are demonstrated for the 4-bladed UH-60A helicopter. Deviations of computed data from flight data are evaluated. Fourier analysis post-processing that is suitable for aeroelastic stability computations are performed.

Guruswamy, Guru P.↗
NASA NTRS1980

Evaluation of SLAR and thematic mapper MSS data for forest cover mapping using computer-aided analysis techniques

Several possibilities were considered for defining the data set in which the same test areas could be used for each of the four different spatial resolutions being evaluated. The LARSYS CLUSTER was used to sort the vectors into spectral classes to reduce the within-spectral class variability in an effort to develop training statistics. A data quality test was written to determine the basic signal to noise characteristics within the data set being used. Because preliminary analysis of the LANDSAT MSS data revealed the presence of high cirrus clouds, other data sets are being sought.

Hoffer, R. M.↗
NASA NTRS1972

Data processing 1: Advancements in machine analysis of multispectral data

Multispectral data processing procedures are outlined beginning with the data display process used to accomplish data editing and proceeding through clustering, feature selection criterion for error probability estimation, and sample clustering and sample classification. The effective utilization of large quantities of remote sensing data by formulating a three stage sampling model for evaluation of crop acreage estimates represents an improvement in determining the cost benefit relationship associated with remote sensing technology.

Swain, P. H.↗
NASA NTRS1974

Notes for the improvement of the spatial and spectral data classification method

This report examines the spatial and spectral clustering technique for the unsupervised automatic classification and mapping of earth resources satellite data, and makes theoretical analysis of the decision rules and tests in order to suggest how the method might best be applied to other flight data such as Skylab and Spacelab.

Dalton, C. C.↗
NASA NTRS1987

Detection of soil erosion with Thematic Mapper (TM) satellite data within Pinyon-Juniper woodlands

Pinyon-Juniper woodlands dominate approximately 24.3 million hectares (60 million acres) in the western United States. The overall objective was to test the sensitivity of the LANDSAT Thematic Mapper (TM) spectral data for detecting varying degrees of soil erosion within the Pinyon-Juniper woodlands. A second objective was to assess the potential of the spectral data for assigning the Universal Soil Loss Equation (USLE) crop management (C) factor values to varying cover types within the woodland. Thematic Mapper digital data for June 2, 1984 on channels 2, 3, 4, and 5 were used. Digital data analysis was performed using the ELAS software package. Best results were achieved using CLUS, an unsupervised clustering algorithm. Fifteen of the 40 Pinyon-Juniper signatures were identified as being relatively pure Pinyon-Juniper woodland. Final analysis resulted in the grouping of the 15 signatures into three major groups. Ten study sites were selected from each of the three groups and located on the ground. At each site the following field measurements were taken: percent tree canopy and percent understory cover, soil texture, total soil loss, and soil erosion rate estimates. A technique for measuring soil erosion within Pinyon-Juniper woodlands was developed. A theoretical model of site degradation after Pinyon-Juniper invasion is presented.

Price, Kevin Paul↗
NASA NTRS2000

Gamma Ray Emission from Chaotic Winds of Massive Stars

The purpose of this proposal was to search for gamma-ray emission from the winds of hot, massive stars. According to our theoretical calculations, shocks in the winds of massive stars accelerate particles to high energies. The high-energy particles emit synchrotron radio emission (observed by ground-based radio telescopes) and high-energy gamma-ray emission that we predicted should be detectable by the EGRET instrument on the Compton Gamma Ray Observatory between 100 MeV and a few GeV. We obtained EGRET from phases 1, 2, and 3 of the Cygnus OB2 association, a cluster of massive, young stars, to search for this gamma-ray emission. The data products and analysis show a source consistent with the position of Cyg OB2 with approximately the predicted count rate and spectrum.

White, Richard L.↗
NASA NTRS2022

Scraping Unstructured Data to Explore the Relationship between Rainfall Anomalies and Vector-Borne Disease Outbreaks

According to the World Health Organization (WHO), vector-borne diseases such as malaria and dengue account for 17% of all infectious disease cases and lead to more than 700,000 deaths per year. Tracking and predicting the spread of vector-borne diseases is a vital task that could save hundreds of thousands of lives annually. Oftentimes, the first reports of vector-borne disease outbreaks occur through emails and online reporting systems long before they are officially documented. Tracking and predicting the emergence and spread of vector-borne disease outbreaks requires extracting data from these unstructured sources in combination with historical weather and climate data to understand the underlying background triggers and disease dynamics. In this work, we develop a data extraction pipeline for the online outbreak reporting website ProMED-mail that utilizes a web scraper, transformer neural network summarizer, and named entity recognizer to obtain a dataset of malaria, dengue, zika, and chikungunya outbreaks over the last 30 years. This scraped dataset was further analyzed in association with global rainfall anomalies derived from NASA’s Integrated Multi-satellitE Retrievals for GPM [Global Precipitation Mission] (IMERG) dataset. This preliminary analysis was to understand the effect of global rainfall patterns on the spread of vector-borne diseases. Analysis of the ProMED-mail and GPM data shows that vector-borne disease outbreaks are clustered towards the tropics and outbreaks are often amplified during the rainy seasons. Our scraped dataset can be a valuable tool in creating comprehensive georeferenced disease records for modeling and predicting future outbreaks.

Web scraping↗
NASA NTRS2019

Lightning Strike Distance Distribution Beyond a Preexisting Lightning Area

The 45th Weather Squadron (45 WS) asked the Applied Meteorology Unit (AMU) to review the 30-year-old, lightning stand-off distances of 5 nautical miles (nmi) for applicability to today's operations. This was based on the realization that previous lightning strike distance studies did not match how 45 WS issues lightning warnings (Roeder, 2008). The previous lightning distance studies were from the point of origin of the lightning or the average starting location that would tend to be in the core of the thunderstorm. However, the 45 WS issues lightning warnings based on the edge of a preexisting lightning area. Before beginning the AMU project, it took several years to develop a method to calculate a distance distribution beyond a preexisting area (Roeder, 2015). The AMU pulled Lightning Detection and Ranging (LDAR) sensor data from 1/1/2013 to 12/31/2013. This dataset consisted of 37 million individual source data points from the LDAR sensors. Only sources within 50 km north, south, east or west of the LDAR grid center were included in the dataset. This limited the use of LDAR data to that with the greatest accuracy of source detection and increased data processing. Points were grouped into flashes based on spatial and temporal criteria. Based on the sensitivity analysis the AMU performed on the flash clustering algorithm, a time value of 0.3 seconds was found to model flashes adequately. Distance parameters were tested from 1,500 to 7,500 meters (m) in 500 m increments. Distance parameters of both 3,000 m and 4,000 m produced results in the plotting tool that were most representative of the physical behavior of lightning. Thus statistics were gathered for the most representative of these spatial and temporal criteria on the flash size and the polygon expansion distance in order to find the correct data distributions. The best fit curves for the LDAR polygon expansion frequency vs. distances for both the 3 kilometer (km) and 4 km distance threshold values were exponential decay functions and had R2 values of > 0.998, indicating good model fits. The equations of the best fit curves were then used to calculate a desired safety radius of 4 nmi for either 3 km or 4 km distance threshold criteria. The AMU analysis concludes the safe reduction of the 5 nmi lightning warning circles to 4 nmi should improve the operational impact by 36% if based on distance from the center of the property area being protected. If based on the edge of the property being protected, then the reduction is 4.5 nmi to 4 nmi and the operational impact is decreased by 21%. For the 6 nmi lightning warning circles, the recommended 4 nmi stand-off distance will result in a safe reduction of operational impact of 31% if based on the center of the area being protected, or 16% if based on the edge of the property.

Flash clustering algorithm↗