Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “robust clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Landslides in West Coast Metropolitan Areas: The Role of Extreme Weather Events

Rainfall-induced landslides represent a pervasive issue in areas where extreme rainfall intersects complex terrain. A farsighted management of landslide risk requires assessing how landslide hazard will change in coming decades and thus requires, inter alia, that we understand what rainfall events are most likely to trigger landslides and how global warming will affect the frequency of such weather events. We take advantage of 9 years of landslide occurrence data compiled by collating Google news reports and of a high-resolution satellite-based daily rainfall data to investigate what weather triggers landslide along the West Coast US. We show that, while this landslide compilation cannot provide consistent and widespread monitoring everywhere, it captures enough of the events in the major urban areas that it can be used to identify the relevant relationships between landslides and rainfall events in Puget Sound, the Bay Area, and greater Los Angeles. In all these regions, days that recorded landslides have rainfall distributions that are skewed away from dry and low-rainfall accumulations and towards heavy intensities. However, large daily accumulation is the main driver of enhanced hazard of landslides only in Puget Sound. There, landslide are often clustered in space and time and major events are primarily driven by synoptic scale variability, namely "atmospheric rivers" of high humidity air hitting anywhere along the West Coast, and the interaction of frontal system with the coastal orography. The relationship between landslide occurrences and daily rainfall is less robust in California, where antecedent precipitation (in the case of the Bay area) and the peak intensity of localized downpours at sub-daily time scales (in the case of Los Angeles) are key factors not captured by the same-day accumulations. Accordingly, we suggest that the assessment of future changes in landslide hazard for the entire the West Coast requires consideration of future changes in the occurrence and intensity of atmospheric rivers, in their duration and clustering, and in the occurrence of short-duration (sub-daily) extreme rainfall as well. Major regional landslide events, in which multiple occurrences are recorded in the catalog for the same day, are too rare to allow a statistical characterization of their triggering events, but a case study analysis indicates that a variety of synoptic-scale events can be involved, including not only atmospheric rivers but also broader cold- and warm-front precipitation. That a news-based catalog of landslides is accurate enough to allow the identification of different landslide/ rainfall relationships in the major urban areas along the US West Coast suggests that this technology can potentially be used for other English-language cities and could become an even more powerful tool if expanded to other languages and non-traditional news sources, such as social media.

landslides↗

Robust Dark Energy Constraints with the Dark Energy Spectroscopic Survey (Final Technical Report)

This project developed and applied advanced theoretical, computational, and data-analysis methodologies to extract robust and precise cosmological constraints from the Dark Energy Spectroscopic Instrument (DESI). The work focused on maximizing the scientific return of DESI through optimized survey strategy, novel higher-order clustering statistics, improved modeling of small-scale structure, and rigorous mitigation of observational systematics. Over the award period, the project made substantial contributions to DESI science planning, produced new methods for bispectrum and three-point correlation function analyses, advanced constraints on primordial non-Gaussianity, and delivered widely used software tools. The project also played a major role in training graduate students and a postdoctoral researcher who contributed directly to DESI key projects. The results have significantly enhanced the cosmological reach of DESI and provide a strong foundation for future surveys such as DESI-II and Stage-V experiments.

79 ASTRONOMY AND ASTROPHYSICS↗

Fuzzy logic, neural networks, and soft computing

The past few years have witnessed a rapid growth of interest in a cluster of modes of modeling and computation which may be described collectively as soft computing. The distinguishing characteristic of soft computing is that its primary aims are to achieve tractability, robustness, low cost, and high MIQ (machine intelligence quotient) through an exploitation of the tolerance for imprecision and uncertainty. Thus, in soft computing what is usually sought is an approximate solution to a precisely formulated problem or, more typically, an approximate solution to an imprecisely formulated problem. A simple case in point is the problem of parking a car. Generally, humans can park a car rather easily because the final position of the car is not specified exactly. If it were specified to within, say, a few millimeters and a fraction of a degree, it would take hours or days of maneuvering and precise measurements of distance and angular position to solve the problem. What this simple example points to is the fact that, in general, high precision carries a high cost. The challenge, then, is to exploit the tolerance for imprecision by devising methods of computation which lead to an acceptable solution at low cost. By its nature, soft computing is much closer to human reasoning than the traditional modes of computation. At this juncture, the major components of soft computing are fuzzy logic (FL), neural network theory (NN), and probabilistic reasoning techniques (PR), including genetic algorithms, chaos theory, and part of learning theory. Increasingly, these techniques are used in combination to achieve significant improvement in performance and adaptability. Among the important application areas for soft computing are control systems, expert systems, data compression techniques, image processing, and decision support systems. It may be argued that it is soft computing, rather than the traditional hard computing, that should be viewed as the foundation for artificial intelligence. In the years ahead, this may well become a widely held position.

Zadeh, Lofti A.↗

HighDimMixedModels.jl: Robust high-dimensional mixed-effects models across omics data

High-dimensional mixed-effects models are an increasingly important form of regression in which the number of covariates rivals or exceeds the number of samples, which are collected in groups or clusters. The penalized likelihood approach to fitting these models relies on a coordinate descent algorithm that lacks guarantees of convergence to a global optimum. Here, we empirically study the behavior of this algorithm on simulated and real examples of three types of data that are common in modern biology: transcriptome, genome-wide association, and microbiome data. Our simulations provide new insights into the algorithm’s behavior in these settings, and, comparing the performance of two popular penalties, we demonstrate that the smoothly clipped absolute deviation (SCAD) penalty consistently outperforms the least absolute shrinkage and selection operator (LASSO) penalty in terms of both variable selection and estimation accuracy across omics data. To empower researchers in biology and other fields to fit models with the SCAD penalty, we implement the algorithm in a Julia package, HighDimMixedModels.jl .

Gorstein, Evan↗

Unitary coupled-cluster based self-consistent polarization propagator theory: A quadratic unitary coupled-cluster singles and doubles scheme

The development of a quadratic unitary coupled-cluster singles and doubles (qUCCSD) based self-consistent polarization propagator method is reported here. We present a simple strategy for truncating the commutator expansion of the unitary version of coupled-cluster transformed Hamiltonian $\bar{H}$. The qUCCSD method for the electronic ground state includes up to double commutators for the amplitude equations and up to cubic commutators for the energy expression. The qUCCSD excited-state eigenvalue equations include up to double commutators for the singles–singles block of $\bar{H}$, single commutators for the singles–doubles and doubles–singles blocks, and the bare Hamiltonian for the doubles–doubles block. Benchmark qUCCSD calculations of the ground-state properties and excitation energies for representative molecules demonstrate significant improvement of the accuracy and robustness over the previous UCC3 scheme derived using Møller–Plesset perturbation theory.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Automated Identification of Characteristic Droplet Size Distributions in Stratocumulus Clouds Utilizing a Data Clustering Algorithm

Abstract Droplet-level interactions in clouds are often parameterized by a modified gamma fitted to a “global” droplet size distribution. Do “local” droplet size distributions of relevance to microphysical processes look like these average distributions? This paper describes an algorithm to search and classify characteristic size distributions within a cloud. The approach combines hypothesis testing, specifically, the Kolmogorov–Smirnov (KS) test, and a widely used class of machine learning algorithms for identifying clusters of samples with similar properties: density-based spatial clustering of applications with noise (DBSCAN) is used as the specific example for illustration. The two-sample KS test does not presume any specific distribution, is parameter free, and avoids biases from binning. Importantly, the number of clusters is not an input parameter of the DBSCAN-type algorithms but is independently determined in an unsupervised fashion. As implemented, it works on an abstract space from the KS test results, and hence spatial correlation is not required for a cluster. The method is explored using data obtained from the Holographic Detector for Clouds (HOLODEC) deployed during the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. The algorithm identifies evidence of the existence of clusters of nearly identical local size distributions. It is found that cloud segments have as few as one and as many as seven characteristic size distributions. To validate the algorithm’s robustness, it is tested on a synthetic dataset and successfully identifies the predefined distributions at plausible noise levels. The algorithm is general and is expected to be useful in other applications, such as remote sensing of cloud and rain properties. Significance Statement A typical cloud can have billions of drops spread over tens or hundreds of kilometers in space. Keeping track of the sizes, positions, and interactions of all of these droplets is impractical, and, as such, information about the relative abundance of large and small drops is typically quantified with a “size distribution.” Droplets in a cloud interact locally, however, so this work is motivated by the question of whether the cloud droplet size distribution is different in different parts of a cloud. A new method, based on hypothesis testing and machine learning, determines how many different size distributions are contained in a given cloud. This is important because the size distribution describes processes such as cloud droplet growth and light transmission through clouds.

54 ENVIRONMENTAL SCIENCES↗

Regulating Gas Transport in Molecularly Engineered Polymer Membranes (Final Technical Report)

Energy-efficient separation processes are essential for a wide range of applications ranging from clean fuels (e.g., hydrogen purification) and petroleum refining (e.g., natural gas processing) to water purification and carbon capture. Membrane-mediated separations have shown tremendous promise in providing high productivity and high separation efficiency at significantly lower energy consumption, e.g., up to 90% less energy cost than traditional thermally driven processes such as distillation. Polymeric membranes–the dominant separation membrane materials–have yet to reach their full potential due to their limitations in long-term durability (e.g., productivity loss over the period of their lifetime due to physical aging) or insufficient stability under harsh conditions (e.g., high temperature, chemically complex feeds). This research seeks to establish a new paradigm in polymer membrane material design by harnessing crosslinked model networks with well-defined yet finely tailorable microstructure to facilitate fast and selective gas transport and simultaneously enhance membrane stability. Unlike traditional randomly crosslinked polymers, which suffer from structural inconsistencies and consequently suboptimal gas separation performance, crosslinked model network membranes prepared via a precisely controlled end-linking process enables the creation of previously unattainable microstructure tunability, which, in turn, results in versatile crosslinked membranes with high separation performance that meet the needs of various challenging gas separations. Using model network framework as a fundamental tool by applying this concept in diverse polymer categories, this work has led to the development of various innovative crosslinked membrane structures such as unimodal, bimodal and clustered model networks. These advanced crosslinked polymer membranes not only demonstrate exceptional gas separation performance that significantly outperform existing randomly crosslinked membranes, but also possess excellent long-term durability and robust stability under complex operating conditions. From a fundamental perspective, results from this research provide critical mechanistic insights into gas separation in crosslinked polymer membranes, addressing key knowledge gaps and opening new avenues for membrane design to meet various separation needs. The new membrane materials produced from this research enable the use of polymeric membranes for high temperature gas separations, offering substantial energy and cost savings by eliminating the need for repeated cooling-heating cycles in industrial processes.

02 PETROLEUM↗

Fusion of monocular cues to detect man-made structures in aerial imagery

The extraction of buildings from aerial imagery is a complex problem for automated computer vision. It requires locating regions in a scene that possess properties distinguishing them as man-made objects as opposed to naturally occurring terrain features. It is reasonable to assume that no single detection method can correctly delineate or verify buildings in every scene. A cooperative-methods paradigm is useful in approaching the building extraction problem. Using this paradigm, each extraction technique provides information which can be added or assimilated into an overall interpretation of the scene. Thus, the main objective is to explore the development of computer vision system that integrates the results of various scene analysis techniques into an accurate and robust interpretation of the underlying three dimensional scene. The problem of building hypothesis fusion in aerial imagery is discussed. Building extraction techniques are briefly surveyed, including four building extraction, verification, and clustering systems. A method for fusing the symbolic data generated by these systems is described, and applied to monocular image and stereo image data sets. Evaluation methods for the fusion results are described, and the fusion results are analyzed using these methods.

Shufelt, Jefferey↗

Synthesis of Nitro Compounds from Nitrogen Dioxide Captured in a Metal-Organic Framework

Increasing levels of air pollution are driving the need for the development of new processes that take “waste-to-chemicals”. Herein, we report the capture and conversion under ambient conditions of a major air pollutant, NO 2 , using a robust metal-organic framework (MOF) material, Zr-bptc (H 4 bptc = 3,3',5,5'-biphenyltetracarboxylic acid), comprising {Zr 6 (μ 3 -O) 4 (μ 3 -OH) 4 (COO) 12 } clusters linked by 4-connected bptc 4– ligands in an ftw topology. At 298 K, Zr-bptc shows exceptional stability and adsorption of NO 2 at both low (4.9 mmol g –1 at 10 mbar) and high pressures (13.8 mmol g –1 at 1.0 bar), as measured by isotherm experiments. Dynamic breakthrough experiments have confirmed the selective retention of NO 2 by Zr-bptc at low concentrations under both dry and wet conditions. The immobilized NO 2 can be readily transformed into valuable nitro compounds relevant to construction, agrochemical, and pharmaceutical industries. In situ crystallographic and spectroscopic studies reveal strong binding interactions of NO 2 to the {Zr 6 (μ 3 -O) 4 (μ 3 -OH) 4 (COO) 12 } cluster node. This study paves a circular pathway to enable the integration of nitrogen-based air pollutants into the production of fine chemicals.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

MVP: a modular viromics pipeline to identify, filter, cluster, annotate, and bin viruses from metagenomes

While numerous computational frameworks and workflows are available for recovering prokaryote and eukaryote genomes from metagenome data, only a limited number of pipelines are designed specifically for viromics analysis. With many viromics tools developed in the last few years alone, it can be challenging for scientists with limited bioinformatics experience to easily recover, evaluate quality, annotate genes, dereplicate, assign taxonomy, and calculate relative abundance and coverage of viral genomes using state-of-the-art methods and standards. Here, we describe Modular Viromics Pipeline (MVP) v.1.0, a user-friendly pipeline written in Python and providing a simple framework to perform standard viromics analyses. MVP combines multiple tools to enable viral genome identification, characterization of genome quality, filtering, clustering, taxonomic and functional annotation, genome binning, and comprehensive summaries of results that can be used for downstream ecological analyses. Overall, MVP provides a standardized and reproducible pipeline for both extensive and robust characterization of viruses from large-scale sequencing data including metagenomes, metatranscriptomes, viromes, and isolate genomes. As a typical use case, we show how the entire MVP pipeline can be applied to a set of 20 metagenomes from wetland sediments using only 10 modules executed via command lines, leading to the identification of 11,656 viral contigs and 8,145 viral operational taxonomic units (vOTUs) displaying a clear beta-diversity pattern. Further, acting as a dynamic wrapper, MVP is designed to continuously incorporate updates and integrate new tools, ensuring its ongoing relevance in the rapidly evolving field of viromics. MVP is available at https://gitlab.com/ccoclet/mvp and as versioned packages in PyPi and Conda.

59 BASIC BIOLOGICAL SCIENCES↗

Framework for a space shuttle main engine health monitoring system

A framework developed for a health management system (HMS) which is directed at improving the safety of operation of the Space Shuttle Main Engine (SSME) is summarized. An emphasis was placed on near term technology through requirements to use existing SSME instrumentation and to demonstrate the HMS during SSME ground tests within five years. The HMS framework was developed through an analysis of SSME failure modes, fault detection algorithms, sensor technologies, and hardware architectures. A key feature of the HMS framework design is that a clear path from the ground test system to a flight HMS was maintained. Fault detection techniques based on time series, nonlinear regression, and clustering algorithms were developed and demonstrated on data from SSME ground test failures. The fault detection algorithms exhibited 100 percent detection of faults, had an extremely low false alarm rate, and were robust to sensor loss. These algorithms were incorporated into a hierarchical decision making strategy for overall assessment of SSME health. A preliminary design for a hardware architecture capable of supporting real time operation of the HMS functions was developed. Utilizing modular, commercial off-the-shelf components produced a reliable low cost design with the flexibility to incorporate advances in algorithm and sensor technology as they become available.

Hawman, Michael W.↗

Spherical Harmonic Analysis of Particle Velocity Distribution Function: Comparison of Moments and Anisotropies using Cluster Data

This paper presents a spherical harmonic analysis of the plasma velocity distribution function using high-angular, energy, and time resolution Cluster data obtained from the PEACE spectrometer instrument to demonstrate how this analysis models the particle distribution function and its moments and anisotropies. The results show that spherical harmonic analysis produced a robust physical representation model of the velocity distribution function, resolving the main features of the measured distributions. From the spherical harmonic analysis, a minimum set of nine spectral coefficients was obtained from which the moment (up to the heat flux), anisotropy, and asymmetry calculations of the velocity distribution function were obtained. The spherical harmonic method provides a potentially effective "compression" technique that can be easily carried out onboard a spacecraft to determine the moments and anisotropies of the particle velocity distribution function for any species. These calculations were implemented using three different approaches, namely, the standard traditional integration, the spherical harmonic (SPH) spectral coefficients integration, and the singular value decomposition (SVD) on the spherical harmonic methods. A comparison among the various methods shows that both SPH and SVD approaches provide remarkable agreement with the standard moment integration method.

Gurgiolo, Chris↗

Dark Energy Survey Year 6 Results: improved mitigation of spatially varying observational systematics with masking

As photometric surveys reach unprecedented statistical precision, systematic uncertainties increasingly dominate large-scale structure probes relying on galaxy number density. Defining the final survey footprint is critical, as it excludes regions affected by artefacts or suboptimal observing conditions. For galaxy clustering, spatially varying observational systematics, such as seeing, are a leading source of bias. Template maps of contaminants are used to derive spatially dependent corrections, but extreme values may fall outside the applicability range of mitigation methods, compromising correction reliability. The complexity and accuracy of systematics modelling depend on footprint conservativeness, with aggressive masking enabling simpler, robust mitigation. We present a unified approach to define the DES Year 6 joint footprint, integrating observational systematics templates and artefact indicators that degrade mitigation performance. This removes extreme values from an initial seed footprint, leading to the final joint footprint. By evaluating the DES Year 6 lens sample MagLim++ plus plus on this footprint, we enhance the Iterative Systematics Decontamination (ISD) method, detecting non-linear systematic contamination and improving correction accuracy. While the mask's impact on clustering is less significant than systematics decontamination, it remains non-negligible, comparable to statistical uncertainties in certain w(theta) scales and redshift bins. Supporting coherent analyses of galaxy clustering and cosmic shear, the final footprint spans 4031.04 deg2, setting the basis for DES Year 6 1x2pt, 2x2pt, and 3x2pt analyses. This work highlights how targeted masking strategies optimise the balance between statistical power and systematic control in Stage-III and -IV surveys.

Rodríguez-Monroy, M. [Madrid, IFT; IJCLab, Orsay]↗

Mitigating imaging systematics for DESI 2024 emission Line Galaxies and beyond

Emission Line Galaxies (ELGs) are one of the main tracers that the Dark Energy Spectroscopic Instrument (DESI) uses to probe the universe. However, they are afflicted by strong spurious correlations between target density and observing conditions known as imaging systematics. In this paper, we present the imaging systematics mitigation applied to the DESI Data Release 1 (DR1) large-scale structure catalogs used in the DESI 2024 cosmological analyses. We also explore extensions of the fiducial treatment. This includes a combined approach, through forward image simulations (Obiwan) in conjunction with neural network-based regression, to obtain an angular selection function that mitigates the imaging systematics observed in the DESI DR1 ELGs target density. We further derive a line of sight selection function from the forward model that removes the strong redshift dependence between imaging systematics and low redshift ELGs. Combining both angular and redshift-dependent systematics, we construct a three-dimensional selection function and assess the impact of all selection functions on clustering statistics. We quantify differences between these extended treatments and the fiducial treatment in terms of the measured 2-point statistics. We find that the results are generally consistent with the fiducial treatment and conclude that the differences are far less than the imaging systematics uncertainty included in DESI 2024 full-shape measurements. We extend our investigation to the ELGs at 0.6 < z < 0.8, i.e., beyond the redshift range (0.8 < z < 1.6) adopted for the DESI clustering catalog, and demonstrate that determining the full three-dimensional selection function is necessary in this redshift range. Our tests showed that all changes are consistent with statistical noise for BAO analyses indicating they are robust to even severe imaging systematics. Specific tests for the full-shape analysis will be presented in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗

Cosmological constraints from the Planck cluster catalogue with DES shear profiles and Chandra observations

We present cosmological constraints from the Planck PSZ2 cosmological cluster sample, using weak-lensing shear profiles from Dark Energy Survey (DES) data and X-ray observations from the Chandra telescope for the mass calibration. We compute hydrostatic mass estimates for all clusters in the PSZ2 sample with a scaling relation between their Sunyaev-Zeldovich signal and X-ray derived hydrostatic mass, calibrated with the Chandra data. We introduce a method to correct these masses with a hydrostatic mass bias using shear profiles from wide-field galaxy surveys. We simultaneously fit the number counts of the PSZ2 sample and the mass calibration with the DES data, finding $Ω_\text{m}=0.312^{+0.018}_{-0.024}$, $σ_8=0.777\pm 0.024$, $S_8\equiv σ_8 \sqrt{Ω_\text{m} / 0.3}=0.791^{+0.023}_{-0.021}$, and $(1-b)=0.844^{+0.055}_{-0.062}$ for our baseline analysis when combined with BAO data. When considering a hydrostatic mass bias evolving with mass, we find $Ω_\text{m}=0.353^{+0.025}_{-0.031}$, $σ_8=0.751\pm 0.023$, and $S_8=0.814^{+0.019}_{-0.020}$. We verify the robustness of our results by exploring a variety of analysis settings, with a particular focus on the definition of the halo centre used for the extraction of shear profiles. We compare our results with a number of other analyses, in particular two recent analyses of cluster samples obtained from SPT and eROSITA data that share the same mass calibration data set. We find that our results are in overall agreement with most late-time probes, in very mild tension with CMB results (1.6$σ$), and in significant tension with results from eROSITA clusters (2.9$σ$). We confirm that our mass calibration is consistent with the eROSITA analysis by comparing masses for clusters present in both Planck and eROSITA samples, eliminating it as a potential cause of tension.

Aymerich, G. [Orsay, IAS; AIM, Saclay] (ORCID:0009↗

Automatic microseismic event picking via unsupervised machine learning

SUMMARY Effective and efficient arrival picking plays an important role in microseismic and earthquake data processing and imaging. Widely used short-term-average long-term-average ratio (STA/LTA) based arrival picking algorithms suffer from the sensitivity to moderate-to-strong random ambient noise. To make the state-of-the-art arrival picking approaches effective, microseismic data need to be first pre-processed, for example, removing sufficient amount of noise, and second analysed by arrival pickers. To conquer the noise issue in arrival picking for weak microseismic or earthquake event, I leverage the machine learning techniques to help recognizing seismic waveforms in microseismic or earthquake data. Because of the dependency of supervised machine learning algorithm on large volume of well-designed training data, I utilize an unsupervised machine learning algorithm to help cluster the time samples into two groups, that is, waveform points and non-waveform points. The fuzzy clustering algorithm has been demonstrated to be effective for such purpose. A group of synthetic, real microseismic and earthquake data sets with different levels of complexity show that the proposed method is much more robust than the state-of-the-art STA/LTA method in picking microseismic events, even in the case of moderately strong background noise.

Chen, Yangkang↗

Gene Expression Dynamics Inspector (GEDI): for integrative analysis of expression profiles

Genome-wide expression profiles contain global patterns that evade visual detection in current gene clustering analysis. Here, a Gene Expression Dynamics Inspector (GEDI) is described that uses self-organizing maps to translate high-dimensional expression profiles of time courses or sample classes into animated, coherent and robust mosaics images. GEDI facilitates identification of interesting patterns of molecular activity simultaneously across gene, time and sample space without prior assumption of any structure in the data, and then permits the user to retrieve genes of interest. Important changes in genome-wide activities may be quickly identified based on 'Gestalt' recognition and hence, GEDI may be especially useful for non-specialist end users, such as physicians. AVAILABILITY: GEDI v1.0 is written in Matlab, and binary Matlab.dll files which require Matlab to run can be downloaded for free by academic institutions at http://www.chip.org/~ge/gedihome.html Supplementary information: http://www.chip.org/~ge/gedihome.html.

Database Management Systems↗