Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analysis cluster”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Clustering earthquake signals and background noises in continuous seismic data with unsupervised deep learning

The continuously growing amount of seismic data collected worldwide is outpacing our abilities for analysis, since to date, such datasets have been analyzed in a human-expert intensive, supervised fashion. Moreover, analyses that are conducted can be strongly biased by the standard models employed by seismologists. In response to both of these challenges, we develop a new unsupervised machine learning framework for detecting and clustering seismic signals in continuous seismic records. Our approach combines a deep scattering network and a Gaussian mixture model to cluster seismic signal segments and detect novel structures. To illustrate the power of the framework, we analyze seismic data acquired during the June 2017 Nuugaatsiaq, Greenland landslide. We demonstrate the blind detection and recovery of the repeating precursory seismicity that was recorded before the main landslide rupture, which suggests that our approach could lead to more informative forecasting of the seismic activity in seismogenic areas.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Unsupervised Segmentation and Clustering Workflow for Efficient Processing of 4D-STEM and 5D-STEM Data

Four-dimensional scanning transmission electron microscopy (4D-STEM) enables mapping of diffraction information with nanometer-scale spatial resolution, offering detailed insight into local structure, orientation, and strain. However, as data dimensionality and sampling density increase, particularly for in situ scanning diffraction experiments (5D-STEM), robust segmentation of structurally consistent behavior across sequential measurements becomes essential for efficient and physically meaningful analysis. Here, we introduce a clustering framework that identifies crystallographically distinct domains from 4D-STEM datasets. By using local diffraction-pattern similarity as a metric, the method extracts closed contours delineating spatially contiguous regions. This approach produces cluster-averaged diffraction patterns that improve signal quality while reducing data volume by orders of magnitude, enabling rapid and accurate orientation, phase, and strain mapping. We demonstrate the applicability of this approach to in situ liquid-cell 4D-STEM data of gold nanoparticle growth. Our method provides a scalable and generalizable route for spatially coherent segmentation, data compression, and quantitative structure–strain mapping across diverse 4D-STEM modalities. The full analysis code and example workflows are publicly available to support reproducibility and reuse.

4D-STEM↗
NASA NTRS1973

The JSC clustering program ISOCLS and its applications

The clustering program ISOCLS developed at the Johnson Space Center, Houston, Texas, has been extensively used in the pattern analysis and classification of remote sensor data collected by aircraft and by the Earth Resources Technology Satellite ERTS-1. This paper discusses the theory behind this clustering algorithm. Several new ideas that have been incorporated in ISOCLS are discussed. Among these are the novel philosophy of operation behind the procedure, which assumes that a population (i.e., a class or a cluster) can be treated as the union of an appropriate number of subpopulations, and the termination of the clustering program by a 'chaining algorithm.' Finally, this paper reports the results of the application of ISOCLS to an investigation on rangeland vegetation mapping using ERTS-1 data.

Kan, E. P.↗
DOE OSTI2023

Mobility Gaps between Low-Income and Not Low-Income Households: A Case Study in New York State

Understanding the travel challenges faced by low-income residents has always been and continues to be one of the most important transportation equity topics. This study aims to explore the mobility gaps between low-income households (HHs) and not low-income HHs, and how the gaps vary within different socio-demographic population groups in New York State (NYS). The latest National Household Travel Survey data was used as the primary data source for the analysis. The study first employed the K-prototype clustering algorithm to categorize the HHs in NYS based on their socio-demographic attributes. Five population groups were identified based on nine different household (HH) features such as HH size, vehicle ownership, and elderly status of its members. Then, the mobility differences, measured by trip frequency, trip distance, travel time, and person miles traveled, were examined among the five population groups. Results suggest that the individuals in low-income HHs consistently took fewer trips and made shorter trips compared to their not low-income counterparts in NYS. The travel distance gaps were most obvious among white HHs with more vehicles than drivers. In addition, while the population from low-income HHs made shorter trips on average (2.7 mi shorter per trip), they experienced longer travel time than those from not low-income HHs (1.8 min longer per trip). These key findings provide a deeper understanding of the travel behavior disparities between low-income and not low-income households. The findings could also support policymakers and transportation planners in addressing the critical needs of residents in low-income households in NYS and provide inputs for designing a more equitable transportation system.

Liu, Yuandong↗
DOE OSTI2025

Visualizing Temporal Topic Embeddings with a Compass

—Dynamic topic modeling is useful at discovering the development and change in latent topics over time. However, present methodology relies on algorithms that separate document and word representations. This prevents the creation of a meaningful embedding space where changes in word usage and documents can be directly analyzed in a temporal context. This paper proposes an expansion of the compass-aligned temporal Word2Vec methodology into dynamic topic modeling. Such a method allows for the direct comparison of word and document embeddings across time in dynamic topics. This enables the creation of visualizations that incorporate temporal word embeddings within the context of documents into topic visualizations. In experiments against the current state-of-the-art, our proposed method demonstrates overall competitive performance in topic relevancy and diversity across temporal datasets of varying size. Simultaneously, it provides insightful visualizations focused on temporal word embeddings while maintaining the insights provided by global topic evolution, advancing our understanding of how topics evolve over time.

Cluster analysis↗
NASA NTRS1989

Land Analysis System (LAS)

Version 4.1 of LAS provides flexible framework for algorithm development and processing and analysis of image data. Over 500,000 lines of code enable image repair, clustering, classification, film processing, geometric registration, radiometric correction, and manipulation of image statistics.

Pease, P. B.↗
NASA NTRS1989

Confirmation of dust clusters in the coma of Comet Halley

Measurements from the Ducma dust counter and mass analyzer on the Vega spacecraft are compared with those from the SP-1 sensor on the same spacecraft. It is shown that the sequence of pulses recorded by the SP-1 instruments show deviations from a random distribution. Some of the nonrandom sequences observed by SP-1 are in the same time intervals as the clusters observed by the Ducma instrument. The results support the analysis of Ducma data which showed highly structured variations of dust intensity through the coma and large fluxes of low mass particles extending to great distances beyond the coma (Simpson et al., 1986).

Simpson, J. A.↗
NASA NTRS1993

The hot interstellar medium in NGC 1399

The first two high signal-to-noise, broad bandpass x-ray spectra of elliptical galaxies were obtained with the Broad Band X-ray Telescope (BBXRT) as part of the December 1990 Astro mission. These observations provided unprecedented information on the thermal and metallicity structure of the hot interstellar media in two ellipticals: NGC 1399, the central galaxy in the Fornax cluster, and NGC 4472, the brightest galaxy in the Virgo cluster. The finalized analysis and interpretation of the approximately 4000 sec of BBXRT data on NGC 1399 is reported.

Loewenstein, Michael↗
NASA NTRS1992

The X-ray spectrum of Abell 665

The X-ray spectrum of the Abell 665 cluster of galaxies has been determined by the Ginga satellite. The spectrum can be well fitted by an isothermal model with kT = 8.26(+0.95; -0.81) keV and metal abundance 0.49 +/- 0.16. Joint analysis with lower energy spectral data from the Einstein Observatory is used to constrain the temperature distribution in the cluster. For a particular form of the polytropic temperature model we find the best fit to be the isothermal case, and we set an upper limit on the polytropic index of 1.35 (99-percent confidence). The virial mass of the cluster has been determined from both the X-ray data and the galaxy velocity dispersion. The mass estimates from the optical and X-ray analyses are fully consistent with each other. Abell 665 is found to be 0.82(+0.41; -0.26) times as massive as Coma if the two clusters are assumed to have the same spatial distribution of virial mass.

Hughes, John P.↗
NASA NTRS2013

Aerodynamic Reconstruction Applied to Parachute Test Vehicle Flight Data Analysis

The aerodynamics, both static and dynamic, of a test vehicle are critical to determining the performance of the parachute cluster in a drop test and for conducting a successful test. The Capsule Parachute Assembly System (CPAS) project is conducting tests of NASA's Orion Multi-Purpose Crew Vehicle (MPCV) parachutes at the Army Yuma Proving Ground utilizing the Parachute Test Vehicle (PTV). The PTV shape is based on the MPCV, but the height has been reduced in order to fit within the C-17 aircraft for extraction. Therefore, the aerodynamics of the PTV are similar, but not the same as, the MPCV. A small series of wind tunnel tests and computational fluid dynamics cases were run to modify the MPCV aerodynamic database for the PTV, but aerodynamic reconstruction of the flights has proven an effective source for further improvements to the database. The acceleration and rotational rates measured during free flight, before parachute inflation but during deployment, were used to con rm vehicle static aerodynamics. A multibody simulation is utilized to reconstruct the parachute portions of the flight. Aerodynamic or parachute parameters are adjusted in the simulation until the prediction reasonably matches the flight trajectory. Knowledge of the static aerodynamics is critical in the CPAS project because the parachute riser load measurements are scaled based on forebody drag. PTV dynamic damping is critical because the vehicle has no reaction control system to maintain attitude - the vehicle dynamics must be understood and modeled correctly before flight. It will be shown here that aerodynamic reconstruction has successfully contributed to the CPAS project.

Cassady, Leonard D.↗
NASA NTRS2016

System and Method for Outlier Detection via Estimating Clusters

An efficient method and system for real-time or offline analysis of multivariate sensor data for use in anomaly detection, fault detection, and system health monitoring is provided. Models automatically derived from training data, typically nominal system data acquired from sensors in normally operating conditions or from detailed simulations, are used to identify unusual, out of family data samples (outliers) that indicate possible system failure or degradation. Outliers are determined through analyzing a degree of deviation of current system behavior from the models formed from the nominal system data. The deviation of current system behavior is presented as an easy to interpret numerical score along with a measure of the relative contribution of each system parameter to any off-nominal deviation. The techniques described herein may also be used to "clean" the training data.

Iverson, David J.↗
NASA NTRS2011

Fast Image Texture Classification Using Decision Trees

Texture analysis would permit improved autonomous, onboard science data interpretation for adaptive navigation, sampling, and downlink decisions. These analyses would assist with terrain analysis and instrument placement in both macroscopic and microscopic image data products. Unfortunately, most state-of-the-art texture analysis demands computationally expensive convolutions of filters involving many floating-point operations. This makes them infeasible for radiation- hardened computers and spaceflight hardware. A new method approximates traditional texture classification of each image pixel with a fast decision-tree classifier. The classifier uses image features derived from simple filtering operations involving integer arithmetic. The texture analysis method is therefore amenable to implementation on FPGA (field-programmable gate array) hardware. Image features based on the "integral image" transform produce descriptive and efficient texture descriptors. Training the decision tree on a set of training data yields a classification scheme that produces reasonable approximations of optimal "texton" analysis at a fraction of the computational cost. A decision-tree learning algorithm employing the traditional k-means criterion of inter-cluster variance is used to learn tree structure from training data. The result is an efficient and accurate summary of surface morphology in images. This work is an evolutionary advance that unites several previous algorithms (k-means clustering, integral images, decision trees) and applies them to a new problem domain (morphology analysis for autonomous science during remote exploration). Advantages include order-of-magnitude improvements in runtime, feasibility for FPGA hardware, and significant improvements in texture classification accuracy.

Thompson, David R.↗
NASA NTRS2005

Lightning Contribution to Improvement of Passive Microwave Vertical Structure and Rainfall Estimation

Joint observations from the TRMM mission are used to examine the contribution which lightning observations can make towards estimation of vertical structure and rainfall from passive microwave data. Passive microwave (TMI) observations are binned into distinct vertical structure categories based on cluster analysis of radar (PR) vertical profiles and convective/stratiform (C/S) classifiers. TMI rain estimates are high-biased relative to radar for stratiform and mixed convective/stratiform vertical structures and low-biased relative to radar for convective structures. Significant ambiguity exists in the TMI brightness temperature space between midlevel stratiform and convective profile types which are primary contributors to tropical rainfall. The ambiguity is worst for convective profile types with radar echo tops within and just above the mixed-phase region. The ability of TMI data (including all low and high frequency polarized brightness temperatures; 19V, 19H, 21V, 37V, 37H, 85V, 85H) to predict vertical structure is assessed. The incremental benefit of including TMI convective/stratiform (C/S) classifiers (both 85 GHz polarization and 19, 37 and 85 GHz texture-based) and LZS lightning observations is then considered. The use of all parameters (brightness temperatures, TMI C/S classifiers, lightning data, predicted vertical structure) to reduce TMI/PR rainfall estimate scatter is demonstrated. For both vertical structure and rainfall estimation, inclusion of C/S classifiers and lightning observations provide statistical skill improvements but do not completely alleviate critical midlevel profile C/S ambiguity and related rain estimate errors.

Boccippio, Dennis↗
NASA NTRSDate not supplied

Data Quality Challenges for Analysis Ready Data (ARD)

Data quality plays a critical role in research and applications. The Earth Science Information Partners (ESIP) Information Quality Cluster (IQC) defines four aspects of information quality: Science, Product, Stewardship, and Services. The ESIP IQC has become internationally recognized as an authoritative and responsive resource of information and guidance to data producers and distributors on how to implement data quality standards and best practices for their science data systems, datasets, and data/metadata dissemination services. In recent years, cloud computing environments have provided scale-up capabilities such as data archives and services, enabling interdisciplinary science and applications. More value-added products are expected from data service providers, including Analysis Ready Data (ARD). ARD refers to data that has been preprocessed into a form that allows immediate analysis by the end user, processed to a minimum set of requirements and provides interoperability over time and across multiple datasets. Once a dataset has been developed from its original form to produce ARD, what quality characteristics should the derived dataset or ARD possess? Also, is it safe to assume that the quality of the ARD is consistent with the quality of the source data, or are there special attributes to an ARD that would warrant a secondary, independent quality assessment? What provenance (also called “data lineage”) information needs to be included in ARD? It is important to answer these questions, especially given the ease of use of ARD, and the consequent temptation by users to trust ARD without understanding the limitations or possible variations in quality compared to the source data. In this presentation, we will discuss data quality challenges for ARD products and services and introduce IQC for participation.

data quality↗
DOE OSTI2023

Identification and Characterization of ten Escherichia coli Strains Encoding Novel Shiga Toxin 2 Subtypes, Stx2n as Well as Stx2j, Stx2m, and Stx2o, in the United States

The sharing of genome sequences in online data repositories allows for large scale analyses of specific genes or gene families. This can result in the detection of novel gene subtypes as well as the development of improved detection methods. Here, we used publicly available WGS data to detect a novel Stx subtype, Stx2n in two clinical E. coli strains isolated in the USA. During this process, additional Stx2 subtypes were detected; six Stx2j, one Stx2m strain, and one Stx2o, were all analyzed for variability from the originally described subtypes. Complete genome sequences were assembled from short- or long-read sequencing and analyzed for serotype, and ST types. The WGS data from Stx2n- and Stx2o-producing STEC strains were further analyzed for virulence genes pro-phage analysis and phage insertion sites. Nucleotide and amino acid maximum parsimony trees showed expected clustering of the previously described subtypes and a clear separation of the novel Stx2n subtype. WGS data were used to design OMNI PCR primers for the detection of all known stx1 (283 bp amplicon), stx2 (400 bp amplicon), intimin encoded by eae (221 bp amplicon), and stx2f (438 bp amplicon) subtypes. These primers were tested in three different laboratories, using standard reference strains. An analysis of the complete genome sequence showed variability in serogroup, virulence genes, and ST type, and Stx2 pro-phages showed variability in size, gene composition, and phage insertion sites. The strains with Stx2j, Stx2m, Stx2n, and Stx2o showed toxicity to Vero cells. Stx2j carrying strain, 2012C-4221, was induced when grown with sub-inhibitory concentrations of ciprofloxacin, and toxicity was detected. Taken together, these data highlight the need to reinforce genomic surveillance to identify the emergence of potential new Stx2 or Stx1 variants. The importance of this surveillance has a paramount impact on public health. Per our description in this study, we suggest that 2017C-4317 be designated as the Stx2n type-strain.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2000

The X-Ray Core of the Low-Luminosity Radio Galaxy 3C346 and ASCA Spectroscopy to Test BL LAC/Radio Galaxy Unification

Radio galaxies are relatively faint sources for Advanced Spacecraft for Cosmology Astrophysics (ASCA), and so in order to get the best possible results from the observations two things have been necessary, both of which delayed the fast preparation of papers. Firstly, the best possible data screening and background subtraction were necessary to improve the signal-to-noise, and all our several initial analysis trials were discarded in favor of using FTOOLS versions 4.1 and above. Secondly, we found that the ASCA spectra were statistically too poor to discriminate well between non-thermal and thermal models, never mind the mixture of the two which we expected on the basis of our ROSAT spatial separation of components in radio galaxies. This means that in each case we have needed to combine the ASCA spectroscopy with analysis of data from other X-ray or radio observations in order to exploit the ASCA data to the full. Our analysis for 3C 346 has yielded the cleanest final result. This powerful radio galaxy at a redshift of 0.161, lies in a poor cluster, which we have separated well from the dominant X-ray component of unresolved emission using a spatial analysis of archival ROSAT data. We were then able to fix the thermal component in our ASCA spectral analysis, and have found evidence that the unresolved emission varied by 32 +/- 13% over the 18 months between the ROSAT and ASCA observations. The unresolved X-ray emission does not suffer from intrinsic absorption, and we have related it to radio structures on both milliarcsecond scales and the arcsecond scales which Chandra can resolve. The source is a target of a Chandra AO2 proposal which we have recently submitted to follow up on our ASCA (and ROSAT) work. 3C 346's orientation to the line of sight is uncertain. However, the absence of X-ray absorption, and the radio/optical/X-ray colors, when combined with with previous radio evidence that the source is a foreshortened radio galaxy of the FRII class, suggest that the radio jets are seen at an angle to the line of sight of about 30 deg, intermediate between the radio-galaxy and quasar classes. The relatively hard ASCA response has allowed us to place an upper limit of 5.6 x 10(exp 43) ergs/ s on the 2-10 keV luminosity of any central X-ray component absorbed bN, gas which might be obscuring the broad-line emission region. Attached to this report is an almost final draft of a paper which we have prepared for submission to the Astrophysical Journal. Our combined ASCA and ROSAT results for NGC 6251 rule out our previously preferred flat-spectrum model and inverse-Compton interpretation for the source based on ROSAT data alone, but a softer X-ray spectrum and moderate absorption bring all the available data (including our early VLA HI measurements) into consistency, and we are reasonably confident that we understand the processes responsible for the X-ray emission. We have made some more sensitive HI absorption measurements which are currently being analyzed, and our plans are to publish our ASCA analysis in conjunction with the new HI results. The ASCA data for NGC 4261 have been difficult to interpret. A re-analysis of our ROSAT data with a wider range of physical parameters brings the ROSAT and ASCA results into reasonable agreement only if the emission from hot gas dominates more than suggested by our earlier work, which is itself unexpected since the radio core is bright and a large jet-related X-ray component would bring the source into agreement with results for others of its type. However, we have recently received our Chandra A01 data for this source, with the spatial resolution which allows us to separate thermal and non-thermal emission components. Our ASCA results will be re-interpreted once the analysis of our Chandra data is complete. The interpretation of the ASCA data for BL Lac object 3C 371 is ongoing, in conjunction with analysis of archival multifrequency data. Radio galaxies are complex in their X-ray properties, and hindsight has shown that the spatial resolution of ASCA is too poor for a reliable interpretation of the data without drawing on other observations. However, the ASCA spectra have made a useful contribution to the interpretation of these sources, and the groundwork is now there for more sensitive work using Chandra and XMM-Newton.

Worrall, Diana↗
NASA NTRS1996

Parallel NPARC: Implementation and Performance

Version 3 of the NPARC Navier-Stokes code includes support for large-grain (block level) parallelism using explicit message passing between a heterogeneous collection of computers. This capability has the potential for significant performance gains, depending upon the block data distribution. The parallel implementation uses a master/worker arrangement of processes. The master process assigns blocks to workers, controls worker actions, and provides remote file access for the workers. The processes communicate via explicit message passing using an interface library which provides portability to a number of message passing libraries, such as PVM (Parallel Virtual Machine). A Bourne shell script is used to simplify the task of selecting hosts, starting processes, retrieving remote files, and terminating a computation. This script also provides a simple form of fault tolerance. An analysis of the computational performance of NPARC is presented, using data sets from an F/A-18 inlet study and a Rocket Based Combined Cycle Engine analysis. Parallel speedup and overall computational efficiency were obtained for various NPARC run parameters on a cluster of IBM RS6000 workstations. The data show that although NPARC performance compares favorably with the estimated potential parallelism, typical data sets used with previous versions of NPARC will often need to be reblocked for optimum parallel performance. In one of the cases studied, reblocking increased peak parallel speedup from 3.2 to 11.8.

Townsend, S. E.↗
DOE OSTI2025

Predicting Dynamic-to-Static Correction Factor from Petrophysical Data and Chemostratigraphy using Unsupervised Machine Learning

Estimating static mechanical properties of stratigraphic layers is critical for optimizing subsurface engineering applications. To estimate dynamic-to-static correction factor F ds (static-to-dynamic Young’s modulus ratio) across the Caney shale interval in Oklahoma, USA, we integrated triaxial test measurements and petrophysical data, including well logs and X-ray fluorescence (XRF) using unsupervised machine learning (ML). We used a novel workflow that includes principal component analysis (PCA) to reduce data set dimensionality of well logs and XRF data sets—both separately and combined—creating three scenarios, and later applied inverse distance weighting (IDW) to derive F ds profiles for these scenarios. Furthermore, we applied K-means clustering on each scenario to predict depositional facies, and built a stiffness zonation profile through chemostratigraphic analysis of the terrigenous elements to validate the predicted F ds . The predicted F ds profile from each scenario using the PCA-IDW method was compared with the constant F ds approach from our previous study by calculating the root mean square error (RMSE). The combined data sets scenario yielded the lowest RMSE value of 0.113, while the RMSE values for the well logs and XRF scenarios were 0.131 and 0.129, respectively. In addition, the predicted F ds from the XRF scenario well-matched the stiffness zonation from the chemostratigraphic analysis that was built using the optimized K-means clustering of nine clusters for that scenario. These methods and findings offer a valuable tool for refining lithological classification and improving the F ds profile, potentially enhancing drilling and stimulation strategies for subsurface energy engineering applications.

clastic rock↗