Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,513 records · Page 84
All sourcesNASA NTRSDOE OSTI
NASA NTRS1996

Parallel NPARC: Implementation and Performance

Version 3 of the NPARC Navier-Stokes code includes support for large-grain (block level) parallelism using explicit message passing between a heterogeneous collection of computers. This capability has the potential for significant performance gains, depending upon the block data distribution. The parallel implementation uses a master/worker arrangement of processes. The master process assigns blocks to workers, controls worker actions, and provides remote file access for the workers. The processes communicate via explicit message passing using an interface library which provides portability to a number of message passing libraries, such as PVM (Parallel Virtual Machine). A Bourne shell script is used to simplify the task of selecting hosts, starting processes, retrieving remote files, and terminating a computation. This script also provides a simple form of fault tolerance. An analysis of the computational performance of NPARC is presented, using data sets from an F/A-18 inlet study and a Rocket Based Combined Cycle Engine analysis. Parallel speedup and overall computational efficiency were obtained for various NPARC run parameters on a cluster of IBM RS6000 workstations. The data show that although NPARC performance compares favorably with the estimated potential parallelism, typical data sets used with previous versions of NPARC will often need to be reblocked for optimum parallel performance. In one of the cases studied, reblocking increased peak parallel speedup from 3.2 to 11.8.

Townsend, S. E.↗
NASA NTRS1998

A Survey of Variable Extragalactic Sources with XTE's All Sky Monitor (ASM)

The original goal of the project was the near real-time detection of AGN utilizing the SSC 3 of the ASM on XTE which does a deep integration on one 100 square degree region of the sky. While the SSC never performed sufficiently well to allow the success of this goal, the work on the project has led to the development of a new analysis method for coded aperture systems which has now been applied to ASM data for mapping regions near clusters of galaxies such as the Perseus Cluster and the Coma Cluster. Publications are in preparation that describe both the new method and the results from mapping clusters of galaxies.

Jernigan, Garrett↗
NASA NTRS2002

Spatiotemporal Variability and Propagation of Equatorial Noise Observed by Cluster

We report a multipoint case study of the electromagnetic equatorial noise observed by the Cluster project. High-resolution data were measured in three close points in space located in the morning sector of the outer plasmasphere. We demonstrate a narrow latitudinal extent of the emissions with a typical width of 2 degrees, centered near the minimum-B equator. Power spectra recorded by the different satellites show a complex structure of emission lines whose relative intensities and positions vary at timescales of 1-2 min and/or at spatial scales of tens of wavelengths. These lines do not match harmonics of the local proton cyclotron frequency, as it would be expected if the waves are generated by energetic ions and observed near the source region. We bring observational evidence that the waves propagate with a significant radial component and thus can propagate from a distant generation region located at different radial distances where ion cyclotron frequencies match the observed fine structure.

Santolik, O.↗
NASA NTRS2010

Cascade and Dissipation of Solar Wind Turbulence at Electron Scales: Whistlers or Kinetic Alfv\'en Waves?

Over the past few decades, large-scales solar wind (SW) turbulence has been studied extensively, both theoretically and observationally. Observed power spectra of the low frequency turbulence, which can be described in the magnetohydrodynamic (MHD) limit, are shown to obey the Kolmogorov scaling, $k"{ -5/3 }$, down the local proton gyrofrequency ($C{ci} \sim O.l$-Hz). Turbulence at frequencies above $C{ci}$ has not been thoroughly investigated and remains far less well understood. Above $C{ ci}$ the spectrum steepens to $\sim f"{ -2.5}$ and a debate exists as to whether the turbulence has become dominated by dispersive kinetic Alfven waves (KA W) or by whistler waves, before it is dissipated at small scales, In a case study Sahraoui et al., PRL (2009) have reported the first direct determination of the dissipation range of solar wind turbulence near the electron gyroscale using the high resolution Cluster magnetic and electric field data (up to $10"2$-Hz in the spacecraft reference frame). Above the Doppler-shifted proton scale $C{\rho i}$ a new inertial range with a scaling $\sim f"{ -2.3}$ has been evidenced and shown to remarkably agree with theoretical predictions of a quasi-two-dimensional cascade into KA W turbulence. Here, we use a wider sample of data sets of small scale SW turbulence under different plasma conditions, and investigate under which physical criteria the KA W (or the whistler) turbulence may be observed to carry out the cascade at small scales, These new observations/criteria are compared to the predictions on the cascade and the (kinetic) dissipation from the Vlasov theory. Implications of the results on the heating problem of the solar wind will be discussed.

Sahraoui, Fouad↗
DOE OSTI2026

Creating a Training Dataset for Semantic Segmentation of Canal Networks for Irrigation Modernization

Canal infrastructure has provided critical irrigation water to the western United States for over a century. To continue providing vital water resources to the semi-arid West, irrigation systems must undergo maintenance and modernization. Many canal companies are resource-constrained, and because funding opportunities often require detailed knowledge of existing infrastructure, they can struggle to secure financial capital. We address this problem by creating training data for a semantic segmentation deep learning model to map canal networks throughout the western United States. To create a diverse and robust training dataset, we labelled 1-m NAIP imagery with the locations of no canals, wet canals, and dry/vegetated canals. Since creating these datasets is time consuming, we first developed a preprocessing methodology to identify canals within our four study areas. We used NAIP imagery and provided canal centerline data to buffer, standardize, and cluster the imagery, automating the labeling process as much as possible. However, this still required manual cleaning and manual classification of canal type. Challenges arose when canals were interrupted (e.g., road culverts or piped sections) or when nearby features shared similar characteristics (e.g., irrigated fields, trees, and shadows). Combining automated preprocessing with manual refinement produced four detailed canal masks to be used in the semantic segmentation model developed by Richard Tapia.

13 - HYDRO ENERGY↗
DOE OSTI2020

Loss Informed Dispatcher for Battery Systems (LIDBS)

LIDBS takes battery-system operational data as an input. It then cleans the data using density-based clustering methods, forms an initial loss model, and then embeds the loss model into the optimization formulation. LIDBS provides a much more informed dispatch strategy over state-of-the-art approaches.

Latif, Aadil↗
DOE OSTI2014

Diffusion maps, clustering and fuzzy Markov modeling in peptide folding transitions

Using the helix-coil transitions of alanine pentapeptide as an illustrative example, we demonstrate the use of diffusion maps in the analysis of molecular dynamics simulation trajectories. Diffusion maps and other nonlinear data-mining techniques provide powerful tools to visualize the distribution of structures in conformation space. The resulting low-dimensional representations help in partitioning conformation space, and in constructing Markov state models that capture the conformational dynamics. In an initial step, we use diffusion maps to reduce the dimensionality of the conformational dynamics of Ala5. The resulting pretreated data are then used in a clustering step. The identified clusters show excellent overlap with clusters obtained previously by using the backbone dihedral angles as input, with small—but nontrivial—differences reflecting torsional degrees of freedom ignored in the earlier approach. We then construct a Markov state model describing the conformational dynamics in terms of a discrete-time random walk between the clusters. We show that by combining fuzzy C-means clustering with a transition-based assignment of states, we can construct robust Markov state models. This state-assignment procedure suppresses short-time memory effects that result from the non-Markovianity of the dynamics projected onto the space of clusters. In a comparison with previous work, we demonstrate how manifold learning techniques may complement and enhance informed intuition commonly used to construct reduced descriptions of the dynamics in molecular conformation space.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Coreset Clustering on Small Quantum Computers

Many quantum algorithms for machine learning require access to classical data in superposition. However, for many natural data sets and algorithms, the overhead required to load the data set in superposition can erase any potential quantum speedup over classical algorithms. Recent work by Harrow introduces a new paradigm in hybrid quantum-classical computing to address this issue, relying on coresets to minimize the data loading overhead of quantum algorithms. We investigated using this paradigm to perform k-means clustering on near-term quantum computers, by casting it as a QAOA optimization instance over a small coreset. We used numerical simulations to compare the performance of this approach to classical k-means clustering. We were able to find data sets with which coresets work well relative to random sampling and where QAOA could potentially outperform standard k-means on a coreset. However, finding data sets where both coresets and QAOA work well—which is necessary for a quantum advantage over k-means on the entire data set—appears to be challenging.

42 ENGINEERING↗
DOE OSTI2016

GALAXY INFALL BY INTERACTING WITH ITS ENVIRONMENT: A COMPREHENSIVE STUDY OF 340 GALAXY CLUSTERS

To study systematically the evolution of the angular extents of the galaxy, intracluster medium (ICM), and dark matter components in galaxy clusters, we compiled the optical and X-ray properties of a sample of 340 clusters with redshifts <0.5, based on all the available data from the Sloan Digital Sky Survey and Chandra / XMM-Newton . For each cluster, the member galaxies were determined primarily with photometric redshift measurements. The radial ICM mass distribution, as well as the total gravitational mass distribution, was derived from a spatially resolved spectral analysis of the X-ray data. When normalizing the radial profile of galaxy number to that of the ICM mass, the relative curve was found to depend significantly on the cluster redshift; it drops more steeply toward the outside in lower-redshift subsamples. The same evolution is found in the galaxy-to-total mass profile, while the ICM-to-total mass profile varies in an opposite way. The behavior of the galaxy-to-ICM distribution does not depend on the cluster mass, suggesting that the detected redshift dependence is not due to mass-related effects, such as sample selection bias. Also, it cannot be ascribed to various redshift-dependent systematic errors. We interpret that the galaxies, the ICM, and the dark matter components had similar angular distributions when a cluster was formed, while the galaxies traveling in the interior of the cluster have continuously fallen toward the center relative to the other components, and the ICM has slightly expanded relative to the dark matter although it suffers strong radiative loss. This cosmological galaxy infall, accompanied by an ICM expansion, can be explained by considering that the galaxies interact strongly with the ICM while they are moving through it. The interaction is considered to create a large energy flow of 10{sup 4445} erg s{sup 1} per cluster from the member galaxies to their environment, which is expected to continue over cosmological timescales.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Novel Technique for Developing Linearized Convex System Models from Experimentally Derived Data

This paper presents a novel technique for generating a convex system model from an experimentally derived data set which features variance among repeated measurements. The convex system model developed as a test case characterizes the dynamic system losses of a vanadium redox flow battery as a function of the active power output and the battery state of charge. The technique hinges on a pre-cleaning via clustering procedure which precedes the formation of a planar convex hull comprised of triangular simplices. The clustering procedure efficiently reduces the experimental data set while mitigating variance among repeated measurements and removes outliers. Ultimately, the lower evelope of the planar convex hull serves as the desired convex system model. The proposed technique reduces systematic model error which is otherwise present when directly developing a planar convex hull model based on an unreduced data set.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2012

Extracting Operating Modes from Building Electrical Load Data: Preprint

Empirical techniques for characterizing electrical energy use now play a key role in reducing electricity consumption, particularly miscellaneous electrical loads, in buildings. Identifying device operating modes (mode extraction) creates a better understanding of both device and system behaviors. Using clustering to extract operating modes from electrical load data can provide valuable insightsinto device behavior and identify opportunities for energy savings. We present a fast and effective heuristic clustering method to identify and extract operating modes in electrical load data.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2013

Automatic Data Filter Customization Using a Genetic Algorithm

This work predicts whether a retrieval algorithm will usefully determine CO2 concentration from an input spectrum of GOSAT (Greenhouse Gases Observing Satellite). This was done to eliminate needless runtime on atmospheric soundings that would never yield useful results. A space of 50 dimensions was examined for predictive power on the final CO2 results. Retrieval algorithms are frequently expensive to run, and wasted effort defeats requirements and expends needless resources. This algorithm could be used to help predict and filter unneeded runs in any computationally expensive regime. Traditional methods such as the Fischer discriminant analysis and decision trees can attempt to predict whether a sounding will be properly processed. However, this work sought to detect a subsection of the dimensional space that can be simply filtered out to eliminate unwanted runs. LDAs (linear discriminant analyses) and other systems examine the entire data and judge a "best fit," giving equal weight to complex and problematic regions as well as simple, clear-cut regions. In this implementation, a genetic space of "left" and "right" thresholds outside of which all data are rejected was defined. These left/right pairs are created for each of the 50 input dimensions. A genetic algorithm then runs through countless potential filter settings using a JPL computer cluster, optimizing the tossed-out data s yield (proper vs. improper run removal) and number of points tossed. This solution is robust to an arbitrary decision boundary within the data and avoids the global optimization problem of whole-dataset fitting using LDA or decision trees. It filters out runs that would not have produced useful CO2 values to save needless computation. This would be an algorithmic preprocessing improvement to any computationally expensive system.

Mandrake, Lukas↗
DOE OSTI2021

Balanced k -means clustering on an adiabatic quantum computer

Adiabatic quantum computers are a promising platform for efficiently solving challenging optimization problems. Therefore, many are interested in using these computers to train computationally expensive machine learning models. We present a quantum approach to solving the balanced k-means clustering training problem on the D-Wave 2000Q adiabatic quantum computer. In order to do this, we formulate the training problem as a quadratic unconstrained binary optimization (QUBO) problem. Unlike existing classical algorithms, our QUBO formulation targets the global solution to the balanced k-means model. We test our approach on a number of small problems and observe that despite the theoretical benefits of the QUBO formulation, the clustering solution obtained by a modern quantum computer is usually inferior to the solution obtained by the best classical clustering algorithms. Nevertheless, the solutions provided by the quantum computer do exhibit some promising characteristics. We also perform a scalability study to estimate the run time of our approach on large problems using future quantum hardware. Finally, as a final proof of concept, we used the quantum approach to cluster random subsets of the Iris benchmark data set.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Utilization of Machine Learning Techniques for Managing the Tracking and Data Relay Satellite Constellation

National Aeronautics and Space Administration’s (NASA) Goddard Space Flight Center (GSFC) operates a constellation of ten geosynchronous Tracking and Data Relay Satellites (TDRS). The TDRS constellation consists of multiple geosynchronous communication relay satellites located around the equator so they can provide continual coverage of any mission in low earth orbit. The TDRS are located primarily in three oceanic regions around the earth. NASA’s White Sands Complex provides the ground communication support for TDRS located over the Atlantic and Pacific Oceans. Another TDRS ground station in Guam supports the TDRS over the Indian Ocean. With these satellites the TDRS network can provide continuous coverage of satellites in low-earth orbit. The NASA Space Network (SN) project office at GSFC manages the constellation of spacecraft. Major customers of the TDRS constellation include, but are not limited to, the International Space Station and the Hubble Space Telescope. The TDRS constellation has three generations of satellites and has been active for over 30 years providing reliable communication links between customer satellites and corresponding ground stations. However, one of the major concerns for TDRS, and in any space mission, is to ensure the health and safety of the spacecraft. Generally, engineers use telemetry data to monitor and analyze the performance and state of health of the spacecraft. Telemetry data contains hundreds of parameters that monitor each important component in the spacecraft, which can be utilized to recognize and characterize the behavior of the spacecraft. Each parameter contains considerable information to represent time-dependent properties of each spacecraft subsystem and component. During the entire life of a TDRS spacecraft, thousands of gigabytes of telemetry data are transmitted in real-time from the spacecraft to the ground station at the White Sands Complex in Las Cruces, New Mexico, and recorded as historical data sets for engineers to process and analyze the events that occurred on-orbit. These parameters contain the function of multiple spacecraft subsystems, such as the attitude control system (ACS), Thermal, Electrical Power Subsystem (EPS), etc. . The first and second generations have exceeded their required lifetime and NASA is keen to manage these spacecrafts carefully in order to maximize the remaining life using the spacecraft telemetry. The challenge is to know when the risk of losing a spacecraft in geosynchronous orbit exceeds the benefit of continued operations for customer support. In the TDRS fleet, the EPS is the most critical subsystem related to spacecraft operations. Failure of the EPS would strand a spacecraft in geosynchronous orbit. Since EPS provides power to the spacecraft, component failures ultimately lead to the inability to support the spacecraft loads and the communications payload. For instance, TDRS-8 has several anomalies in EPS including the Bus Voltage Limiter (BVL) shunt current, solar array loss of circuits, and failed battery cells. Any of these anomalies can cause critical issues to the spacecraft. Therefore, developing a system to analyze and perform early detection of a potential anomaly is an important issue in telemetry data analysis. In recent years, Telemetry Mining (TM) has been proposed to process telemetry data by using Data Mining (DM) techniques such as classification, clustering, regression and anomaly detection. Anomaly detection, also known as outlier detection, has been widely used in many data mining areas such as remote sensing, medical data processing and digital image processing. The goal of anomaly detection is to detect abnormal data, which contains a relatively low probability of occurrence among the entire data set. Early detection of anomalies is one of the most significant issues in managing the spacecraft configuration. If anomalies can be detected early enough, then the redundant resources can be used to extend the life of the operational spacecraft. We present an unsupervised anomaly detection method to process the EPS data extracted from TDRS-8. This is different from traditional analytical methods, which use telemetry data to illustrate behavior and physical meaning of each spacecraft component. TM connects multiple parameters as a vector and then conducts data analysis on this high dimension telemetry vector. This method is looking at the properties of a high dimensional vector that is able to consider the relationship between different parameters in the anomaly detection problem. This kind of method performs much better than the traditional limit checking method. In addition, we propose a new approach of real-time anomaly detection to process telemetry data in real-time, which can then be applied to spacecraft monitoring with high reliability, low cost and high accuracy.

Machine Learning (ML)↗
NASA NTRSDate not supplied

Clustering Approach To Identifying Low Lunar Frozen Orbits In A High-Fidelity Model

Low lunar frozen orbits are of continued interest in the astrodynamics community for trajectory design and space domain awareness. This paper presents a data-driven approach to analyzing a wide variety of numerically-generated lunar trajectories in a 100 × 100 lunar gravity model with the point mass gravity of the Earth and Sun. First, clustering is used to extract a summary of these trajectories: within each cluster, trajectories possess a geometrically similar evolution of perilune but varying drift and lifetimes. Within some clusters, trajectories with a bounded perilune evolution are also identified to produce candidates for lunar frozen orbits of distinct geometries.

lunar frozen orbit gravity model data clustering↗
DOE OSTI2017

Large-scale Map of Millimeter-wavelength Hydrogen Radio Recombination Lines around a Young Massive Star Cluster

We report the first map of large-scale (10 pc in length) emission of millimeter-wavelength hydrogen recombination lines (mm-RRLs) toward the giant H ii region around the W43-Main young massive star cluster (YMC). Our mm-RRL data come from the IRAM 30 m telescope and are analyzed together with radio continuum and cm-RRL data from the Karl G. Jansky Very Large Array and HCO{sup +} 1–0 line emission data from the IRAM 30 m. The mm-RRLs reveal an expanding wind-blown ionized gas shell with an electron density ∼70–1500 cm{sup −3} driven by the WR/OB cluster, which produces a total Lyα photon flux of 1.5×10{sup 50} s{sup −1}. This shell is interacting with the dense neutral molecular gas in the W43-Main dense cloud. Combining the high spectral and angular resolution mm-RRL and cm-RRL cubes, we derive the two-dimensional relative distributions of dynamical and pressure broadening of the ionized gas emission and find that the RRL line shapes are dominated by pressure broadening (4–55 km s{sup −1}) near the YMC and by dynamical broadening (8–36 km s{sup −1}) near the shell’s edge. Ionized gas clumps hosting ultra-compact H ii regions found at the edge of the shell suggest that large-scale ionized gas motion triggers the formation of new star generation near the periphery of the shell.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Clustering instabilities in sedimenting fluid–solid systems: critical assessment of kinetic-theory-based predictions using direct numerical simulation data

In this paper the quantitative and qualitative ability of a kinetic-theory-based two-fluid model (KT-TFM) is assessed in a state of fully periodic sedimentation (fluidization), with a focus on statistically steady, unstable (clustered) states. The accuracy of KT-TFM predictions is evaluated via direct comparison to direct numerical simulation (DNS) data. The KT-TFM and DNS results span a rather wide parameter space: mean-flow Reynolds numbers on the order of 1 and 10, mean solid volume fractions from 0.1 to 0.4, solid-to-fluid density ratios from 10 to 1000 and elastic and moderately inelastic (restitution coefficient of 0.9) conditions. Data from both KT-TFM and DNS display a rich variety of statistically steady yet unstable structures (clusters). Instantaneous snapshots of KT-TFM and DNS demonstrate remarkable qualitative agreement. This qualitative agreement is quantified by calculating the critical density ratio at which the structure transitions from a chaotic, dynamic state to a regular, plug-flow state, with good overall comparisons. Further quantitative assessments of mean and fluctuating velocities show good agreement at high density ratios but weaker agreement at intermediate to low density ratios depending on the mean-flow Reynolds numbers and solid fractions. Deviations of the KT-TFM results from the DNS data were traced to a breakdown in one of the underlying assumptions of the kinetic theory derivation: high thermal Stokes number. Surprisingly, however, even though the low Knudsen number assumption, also associated with the kinetic theory derivation, is violated throughout most of the parameter space, it does not seem to affect the good quantitative accuracy of KT-TFM simulations.

42 ENGINEERING↗
DOE OSTI2023

Optimizing Performance on Trinity Utilizing Machine Learning, Proxy Applications and Scheduling Priorities

The sheer number of nodes continues to increase in today’s supercomputers, the first half of Trinity alone contains more than 9400 compute nodes. Since the speed of today’s clusters are limited by the slowest nodes, it more important than ever to identify slow nodes, improve their performance if it can be done, and assure minimal usage of slower nodes during performance critical runs. This is an ongoing maintenance task that occurs on a regular basis and, therefore, it is important to minimize the impact upon its users by assessing and addressing slow performing nodes and mitigating their consequences while minimizing down time. These issues can be solved, in large part, through a systematic application of fast running hardware assessment tests, the application of Machine Learning, and making use of performance data to increase efficiency of large clusters. Proxy applications utilizing both MPI and OpenMP were developed to produce data as a substitute for long runtime applications to evaluate node performance. Machine learning is applied to identify underperforming nodes, and policies are being discussed to both minimize the impact of underperforming nodes and increase the efficiency of the system. In this paper, I will describe the process used to produce quickly performing proxy tests, consider various methods to isolate the outliers, and produce ordered lists for use in scheduling to accomplish this task.

97 MATHEMATICS AND COMPUTING↗