Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Ensemble Forecasting/Assimilation/Emissions Estimation with WRF-Chem/DART: Accomplishments, Lessons Learned, and Future Plans

Over the past ten years, we have been conducting research on regional ensemble atmospheric composition forecasting/data assimilation/emissions estimation with WRF-Chem/DART. WRF- Chem/DART integrates the Weather Research and Forecasting model (WRF) with online chemistry (WRF-Chem) into the Data Assimilation Research Testbed (DART). DART is an ensemble data assimilation system based on the ensemble adjustment Kalman filter (EAKF) with adaptive inflation, localization (physical and state space), and an optional non-Gaussian formulation of the EAKF. DART includes assimilation of meteorological and limited chemical observations. WRF-Chem/DART extends DART to include assimilation of: MOPITT CO; IASI CO and O3; MODIS AOD; OMI O3, NO2, and SO2; TROPOMI CO, O3, NO2, and SO2, TES CO, CO2 (research mode), O3, NH3, and CH4 (research mode); CrIS CO, O3, NH3, CH4 (research mode), and PAN; SCIAMACHY NO2; GOME2a NO2; MLS O3 and HNO3; and proxy TEMPO O3, and NO2 satellite retrievals as raw retrievals or as ‘compact phase space retrievals’ (CPSRs) for profile retrievals. WRF-Chem/DART also assimilates in situ atmospheric composition measurements and uses the ‘state augmentation method’ for emissions estimation. In our presentation, we will provide an overview of WRF-Chem/DART: • applications and results; • lessons learned from: (i) independent versus joint assimilation; (ii) total/partial column versus profile retrieval assimilation; (iii) joint in situ and retrieval assimilation; (iv) assimilation at grid resolutions ranging from 100 km to 4 km; (v) dynamic emissions estimation; and • future work related to wildfire emissions estimation and intercomparison of CMAQ (online)/JEDI and CMAQ (online and offline)/DART.

WRF-Chem/DART↗
DOE OSTI2026

Field and Model Data Associated with the Manuscript “Drivers of Streamflow Intermittency in Humid Regions: 1. Evaluating Above- and Below-ground Controls of Flow Persistence in a Forested Catchment”

This package contains field data, modeling files, and scripts supporting the investigation of the drivers of streamflow intermittency in a forested catchment. It includes the field data collected from electrical resistivity tomography (ERT) surveys, ground penetrating radar (GPR), continuous self-potential (SP) monitoring, electromagnetic (EM) imaging, groundwater and stilling well. In addition, it contains the data and results of the coupled water- and electrical-flow model developed using the COMSOL Multiphysics and Advanced Terrestrial Simulator (ATS), as well as software files and Jupyter notebooks used to process the data and generate figures in the manuscript submitted for peer review. The data archive is organized in the following directories: 1) Climate Includes hourly precipitation and daily evapotranspiration time series (2024 – 2025) provided as CSV files, alongside a text file detailing dataset units. 2) Coupled_model Contains two subfolders: Synthetic and Field_Application subfolder. Synthetic subfolder contains the ATS XML input script (can be opened using any code editor) for the four synthetic hydrological cases tested (Connected and gaining, Connected and losing, Disconnected and losing, and dry stream). It also includes other experimental cases to test the influence of precipitation and concentration gradient. For each synthetic case, the flow model simulation is executed using the ATS XML scripts and the included Python script (generate_data_set.py) to convert ATS output to COMSOL-ready input. COMSOL Multiphysics template (.mph can be opened with the commercial software COMSOL and requires a license) is executed using the ATS output data to simulate the potential field. It also includes the Synthetic_model_plot.ipynb (can be opened using any code editor) to visualize the SP result and generate manuscript figures. The data subfolder contains mesh files to run both the ATS (.exo and .stl files can be viewed using Paraview; .h5 files can be opened using HDFView software and h5py Python package) and COMSOL models. Field_Application subfolder contains two subfolders: ES_MDA_inversion and Final_Model. ES_MDA_inversion contains the Python script (.py can be opened using any code editor) and SP observation data used to run the Ensemble Smoother with Multiple Data Assimilation (ES-MDA) inversion sequence to get the optimal model parameters. The Final_model subfolder contains the ATS XML input scripts, data files, output data for the two SP sites. The same workflow steps outlined for the Synthetic subfolder apply here. It also contains the Jupyter notebook (Plot_final_calib.ipynb) to visualize the results of the modeled SP, stream-groundwater exchange and moisture content. 3) Discharge Includes the electrical conductivity (EC) time series (provided as CSV files) from salt slug injections. It also includes the Jupyter notebook (Discharge_process.ipynyb) used to estimate discharge. All discharge measurements collated into rating_curve_processed.csv 4) EM Contains the CSV file of the EM data from the DUALEM-42, including spatial coordinates (x, y, z), apparent conductivity, and in-phase measurements at 2 m coil separations for horizontal coplanar (HCP) and perpendicular (PRP) geometries. 5) ERT Contains raw resistivity data (provided as CSV files), spatial location of each of the electrodes (provided as CSV files), and files used for the resistivity inversion (.resipy can be opened with the open-source ResIPy software). 6) GPR Includes GPR field datasets collected at 100 MHz and 250 MHz antenna frequencies, along with the processing/interpretation project file (GPR_process.gpz can be viewed using EKKO_Project 6, a commercial software by Sensors & Software that requires a license). 7) Slug_test Includes the slug test data at all the groundwater wells provided as CSV files, as well as the Jupyter notebook (Slug_test.ipynb) for calculating hydraulic conductivity. 8) SP Contains the SP data collected in field at the two SP sites (one in the perennial reach and the other in the intermittent reach), provided as DAT files. 9) Well_data Contains two subfolders: 1) Raw, which provides unprocessed pressure, electrical conductivity and temperature timeseries downloaded from the loggers in all the groundwater and stilling wells, and 2) Processed, which contains sorted, QA/QC timeseries data for each well. The data archive also contains data_process.ipynb, a Jupyter notebook used for field data analysis and generating figures (plotting well, SP, climate, and discharge data, as well as calculating head gradient at sites with nested groundwater wells). It also includes DTW.ipynb, a Jupyter notebook containing the code for the dynamic time warping (DTW) with sliding window to evaluate SP signal synchronicity.

ATS↗
NASA NTRS2017

Similarity Assessment of Land Surface Model Outputs in the North American Land Data Assimilation System

Multimodel ensembles are often used to produce ensemble mean estimates that tend to have increased simulation skill over any individual model output. If multimodel outputs are too similar, an individual LSM would add little additional information to the multimodel ensemble, whereas if the models are too dissimilar, it may be indicative of systematic errors in their formulations or configurations. The article presents a formal similarity assessment of the North American Land Data Assimilation System (NLDAS) multimodel ensemble outputs to assess their utility to the ensemble, using a confirmatory factor analysis. Outputs from four NLDAS Phase 2 models currently running in operations at NOAA/NCEP and four new/ upgraded models that are under consideration for the next phase of NLDAS are employed in this study. The results show that the runoff estimates from the LSMs were most dissimilar whereas the models showed greater similarity for root zone soil moisture, snow water equivalent, and terrestrial water storage. Generally, the NLDAS operational models showed weaker association with the common factor of the ensemble and the newer versions of the LSMs showed stronger association with the common factor, with the model similarity increasing at longer time scales. Trade-offs between the similarity metrics and accuracy measures indicated that the NLDAS operational models demonstrate a larger span in the similarity-accuracy space compared to the new LSMs. The results of the article indicate that simultaneous consideration of model similarity and accuracy at the relevant time scales is necessary in the development of multimodel ensemble.

land surface models↗
NASA NTRS2019

Uncertainty in Soil Moisture Retrievals: an Ensemble Approach Using SMOS L-Band Microwave Data

The uncertainty of soil moisture (SM) retrievals from satellite brightness temperature (TB) observations depends primarily on the choice of radiative transfer model (RTM) parameters, prior SM information and TB inputs. This paper studies the sensitivity of several (quasi-)operational and experimental SM retrieval products from the Soil Moisture Ocean Salinity (SMOS) mission to these choices at 11 reference sites, located in 7 watersheds across the United States (US). Different literature-based RTM parameter sets cause large biases between retrievals. Whereas typical RTM parameter sets are calibrated for SM retrievals, it is shown that a parameter set carefully optimized for TB forward modeling can also be used for retrieving SM. It is also shown that the inclusion of dynamic prior SM estimates in a Bayesian retrieval scheme can strongly improve SM retrievals, regardless of the choice of RTM parameters, and that the use of multi-angular and multi-polarization TB does not necessarily lead to superior retrievals compared to retrievals based on TB data at a single incidence angle and polarization. The second part of this paper evaluates ensemble uncertainty metrics for SM retrievals obtained by propagating a wide range of RTM parameters through the RTM. As expected for bounded variables, the spread in the ensemble SM retrievals is smallest for wet and dry SM values and highest for intermediate SM values. After removal of the strong long-term mean bias associated with the RTM parameter values for individual ensemble members, the remaining anomaly ensemble SM spread of 0.037 cu m/cu m approximates the actual time series unbiased root-mean-square-difference of 0.042 cu m/cu m between ensemble mean retrievals and in situ reference data across the reference sites. However, the temporal variability in the anomaly ensemble spread reveals higher-order biases in the retrieval error, which should be accounted for when characterizing retrieval error.

Jan Quets↗
DOE OSTI2025

Simultaneous inference of equation of state parameters and unknown data errors with uncertainty quantification via hierarchical Bayesian posterior maximization

Equations of state (EOSs) are a key component in running hydrodynamic simulations as they relate the thermodynamic states for the material. The Davis reactants EOS is commonly used for modeling high explosives (HEs), and the EOS model parameters are calibrated using material specific data. The calibrations are often performed with uncertainty quantification via Bayesian inference to account for uncertainty in the data and generate ensembles of likely parameters. However, there are relatively few HE data sets to use for calibration and many are historical and lack error information. In this work, we simultaneously calibrate the Davis reactants EOS model parameters and unknown data error terms for the high explosive PBX 9501. To quantify the uncertainty in the models and the data, we use a Bayesian framework for the calibration and compute the hierarchical Bayesian posterior distribution with both a posteriori maximization approach and Markov Chain Monte Carlo. In general, we find that, given our assumptions, the two approaches result in similar calibrated parameters, posterior covariance matrices, and insights about the parameters but that the posterior maximization requires far less computational resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Event Classifications on DNE2 Main Experiment Data using a Convolutional Neural Network Ensemble

The Dynamic Networks (DN) Experiment for FY24 (DNE2) is an experiment within DN with the goal of quantitatively evaluating the effectiveness of solutions developed so far by various researchers under the Low Yield Nuclear Monitoring (LYNM) program using a shared set of metrics and datasets. A key component of this experiment is the mimicking of a signature processing pipeline, and comparing currently accepted and standard-use processing methods to more state-of-the-art processes developed under DN. In this work, we focus specifically on the Event Characterization (EC) Focus Area (FA) of the pipeline, where a seismic event’s magnitude, yield and class are identified. We use Deep Learning (DL) to classify the type of events being processed as either earthquakes (EQs) or explosions (EXs) for three iterations of experiment datasets. The model is noticeably more confident and accurate in classifying explosions than earthquakes, reflecting a known shortcoming of the model, that being of a bias towards predicting explosions over earthquakes in the west coast due to training data biases.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

Observability under recurrent loss of data

An account is given of the concept of extended observability in finite-dimensional linear time-invariant systems under recurrent loss of data, where the state vector has to be reconstructed from an ensemble of sensor data at nonconsecutive samples. An at once necessary and sufficient condition for extended observability that can be expressed via a recursive relation is presented, together with such conditions for this as may be related to the characteristic polynomial of the state transition matrix in a discrete-time setting, or of the system matrix in a continuous-time setting.

Luck, Rogelio↗
NASA NTRS2015

Simultaneous Radar and Satellite Data Storm-Scale Assimilation Using an Ensemble Kalman Filter Approach for 24 May 2011

Assimilating high-resolution radar reflectivity and radial velocity into convection-permitting numerical weather prediction models has proven to be an important tool for improving forecast skill of convection. The use of satellite data for the application is much less well understood, only recently receiving significant attention. Since both radar and satellite data provide independent information, combing these two sources of data in a robust manner potentially represents the future of high-resolution data assimilation. This research combines Geostationary Operational Environmental Satellite 13 (GOES-13) cloud water path (CWP) retrievals with Weather Surveillance Radar-1988 Doppler (WSR-88D) reflectivity and radial velocity to examine the impacts of assimilating each for a severe weather event occurring in Oklahoma on 24 May 2011. Data are assimilated into a 3-km model using an ensemble adjustment Kalman filter approach with 36 members over a 2-h assimilation window between 1800 and 2000 UTC. Forecasts are then generated for 90 min at 5-min intervals starting at 1930 and 2000 UTC. Results show that both satellite and radar data are able to initiate convection, but that assimilating both spins up a storm much faster. Assimilating CWP also performs well at suppressing spurious precipitation and cloud cover in the model as well as capturing the anvil characteristics of developed storms. Radar data are most effective at resolving the 3D characteristics of the core convection. Assimilating both satellite and radar data generally resulted in the best model analysis and most skillful forecast for this event.

Jones, Thomas A.↗
NASA NTRS2022

Kamodo’s Model-Agnostic Satellite Flythrough: Lowering the Utilization Barrier for Heliophysics Model Outputs

Heliophysics model outputs are increasingly accessible, but typically are not usable by the majority of the community unless directly collaborating with the relevant model developers. Prohibitive factors include complex file output formats, cryptic metadata, unspecified and often customized coordinate systems, and non-linear coordinate grids. Some pockets of progress exist, giving interfaces to various simulation outputs, but only for a small set of outputs and typically not with open-source, freely available packages. Additionally, the increasing array of tools built upon these sporadic interfaces are typically model-specific. We present Kamodo’s model-agnostic satellite flythrough capabilities as the solution to the utilization barrier for heliophysics model outputs. Developed at the Community Coordinated Modeling Center, these flythrough capabilities are built in Python upon a network of model-agnostic interfaces developed in collaboration with model developers, providing interpolation results the community can trust. Kamodo’s flythrough capabilities present the user with a growing variety of flythrough tools based upon a rapidly expanding library of heliophysics model outputs in several domains, currently including a variety of Ionosphere-Thermosphere-Mesosphere and global magnetosphere model outputs. Each capability is designed to be easily accessible via simplistic model-agnostic syntax, with the entire package freely available in the cloud on Github. Here, we describe the tools developed, include several sample applications for common science questions, demonstrate interoperability with selected packages, and summarize ongoing developments.

Software↗
NASA NTRS2023

Comment on “Advanced Testing of Low, Medium, and High ECS CMIP6 GCM Simulations Versus ERA5-T2m” by N. Scafetta (2022)

Scafetta (2022, https://doi.org/10.1029/2022gl097716) purports to test Coupled Model Intercomparison Project Phase 6 (CMIP6) climate models through a comparison of temperature changes over three decades. Unfortunately, the paper contains numerous conceptual and statistical errors that undermine all of the conclusions. First, no uncertainty is given for the observational temperature difference, making it impossible to assess compatibility with any model result. Second, the CMIP6 data are the ensemble means for each model, but the metric being tested is sensitive to the internal variability and so the full ensemble for each model must be used. When this is corrected, the conclusion that “all models with ECS > 3.0°C overestimate the observed global surface warming” is not sustained. Third, the statistical test in Section 2 would reject all models even in a perfect model setup given sufficient ensemble members, thus the second conclusion “that spatial t-statistics rejects the data-model agreement” is also not sustainable.

CMIP6↗
NASA NTRS2013

An Ensemble Recentering Kalman Filter with an Application to Argo Temperature Data Assimilation into the NASA GEOS-5 Coupled Model

A two-step ensemble recentering Kalman filter (ERKF) analysis scheme is introduced. The algorithm consists of a recentering step followed by an ensemble Kalman filter (EnKF) analysis step. The recentering step is formulated such as to adjust the prior distribution of an ensemble of model states so that the deviations of individual samples from the sample mean are unchanged but the original sample mean is shifted to the prior position of the most likely particle, where the likelihood of each particle is measured in terms of closeness to a chosen subset of the observations. The computational cost of the ERKF is essentially the same as that of a same size EnKF. The ERKF is applied to the assimilation of Argo temperature profiles into the OGCM component of an ensemble of NASA GEOS-5 coupled models. Unassimilated Argo salt data are used for validation. A surprisingly small number (16) of model trajectories is sufficient to significantly improve model estimates of salinity over estimates from an ensemble run without assimilation. The two-step algorithm also performs better than the EnKF although its performance is degraded in poorly observed regions.

Particle Filter↗
NASA NTRS1994

Wavelet compression techniques for hyperspectral data

Hyperspectral sensors are electro-optic sensors which typically operate in visible and near infrared bands. Their characteristic property is the ability to resolve a relatively large number (i.e., tens to hundreds) of contiguous spectral bands to produce a detailed profile of the electromagnetic spectrum. In contrast, multispectral sensors measure relatively few non-contiguous spectral bands. Like multispectral sensors, hyperspectral sensors are often also imaging sensors, measuring spectra over an array of spatial resolution cells. The data produced may thus be viewed as a three dimensional array of samples in which two dimensions correspond to spatial position and the third to wavelength. Because they multiply the already large storage/transmission bandwidth requirements of conventional digital images, hyperspectral sensors generate formidable torrents of data. Their fine spectral resolution typically results in high redundancy in the spectral dimension, so that hyperspectral data sets are excellent candidates for compression. Although there have been a number of studies of compression algorithms for multispectral data, we are not aware of any published results for hyperspectral data. Three algorithms for hyperspectral data compression are compared. They were selected as representatives of three major approaches for extending conventional lossy image compression techniques to hyperspectral data. The simplest approach treats the data as an ensemble of images and compresses each image independently, ignoring the correlation between spectral bands. The second approach transforms the data to decorrelate the spectral bands, and then compresses the transformed data as a set of independent images. The third approach directly generalizes two-dimensional transform coding by applying a three-dimensional transform as part of the usual transform-quantize-entropy code procedure. The algorithms studied all use the discrete wavelet transform. In the first two cases, a wavelet transform coder was used for the two-dimensional compression. The third case used a three dimensional extension of this same algorithm.

Evans, Bruce↗
DOE OSTI2025

Boosting efficiency and reducing graph reliance: Basis adaptation integration in Bayesian multi-fidelity networks

The computational cost of high-fidelity numerical models makes outer-loop analysis, which requires repeated interrogation of the model such as uncertainty quantification, computationally demanding. Multi-fidelity methods, which construct a surrogate model using data from an ensemble of models of varying cost and accuracy, can substantially reduce the cost of outer-loop analysis. However, these methods can be difficult to apply when the model ensemble does not admit a clear hierarchy a priori and the correlations between models are low. Consequently, in this paper, we present a multi-fidelity method that leverages dimension reduction to enhance the correlation between models, thereby reducing the amount of data needed to train a surrogate from an unordered ensemble of models. Our method utilizes basis adaptation to build low-dimensional polynomial chaos expansions of each model and employs Multi-fidelity Networks to encode the relationships among models. We show that the resulting method exhibit two notable advantages over its counterpart: (1) enhanced accuracy (both reduced bias and variance); and (2) reduced dependency on the graph structure encoding relationships among models. We demonstrate the approach on an analytical test problem and a challenging finite element model for a spent nuclear fuel. Our method produces a surrogate model that is significantly more accurate than either a single-fidelity surrogate or a multi-fidelity surrogate constructed without basis adaptation.

42 ENGINEERING↗
DOE OSTI2024

Ensemble variational Fokker-Planck methods for data assimilation

Particle flow filters solve Bayesian inference problems by smoothly transforming a set of particles into samples from the posterior distribution. Particles move in state space under the flow of an McKean-Vlasov-Itˆo process. This work introduces the Variational Fokker-Planck (VFP) framework for data assimilation, a general approach that includes previously known particle flow filters as special cases. The McKean-Vlasov-Itˆo process that transforms particles is defined via an optimal drift that depends on the selected diffusion term. It is established that the underlying probability density - sampled by the ensemble of particles - converges to the Bayesian posterior probability density. For a finite number of particles the optimal drift contains a regularization term that nudges particles toward becoming independent random variables. Based on this analysis, we derive computationally-feasible approximate regularization approaches that penalize the mutual information between pairs of particles, and avoid particle collapse. Moreover, the diffusion plays a role akin to a particle rejuvenation approach that aims to alleviate particle collapse. The VFP framework is very flexible. Different assumptions on prior and intermediate probability distributions can be used to implement the optimal drift, and localization and covariance shrinkage can be applied to alleviate the curse of dimensionality. A robust implicit-explicit method is discussed for the efficient integration of stiff McKean- Vlasov-Itˆo processes. Here, the effectiveness of the VFP framework is demonstrated on three progressively more challenging test problems, namely the Lorenz ’63, Lorenz ’96 and the quasi-geostrophic equations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2008

Assimilation of GRACE Terrestrial Water Storage Data into a Land Surface Model

The NASA Gravity Recovery and Climate Experiment (GRACE) system of satellites provides observations of large-scale, monthly terrestrial water storage (TWS) changes. In. this presentation we describe a land data assimilation system that ingests GRACE observations and show that the assimilation improves estimates of water storage and fluxes, as evaluated against independent measurements. The ensemble-based land data assimilation system uses a Kalman smoother approach along with the NASA Catchment Land Surface Model (CLSM). We assimilated GRACE-derived TWS anomalies for each of the four major sub-basins of the Mississippi into the Catchment Land Surface Model (CLSM). Compared with the open-loop (no assimilation) CLSM simulation, assimilation estimates of groundwater variability exhibited enhanced skill with respect to measured groundwater. Assimilation also significantly increased the correlation between simulated TWS and gauged river flow for all four sub-basins and for the Mississippi River basin itself. In addition, model performance was evaluated for watersheds smaller than the scale of GRACE observations, in the majority of cases, GRACE assimilation led to increased correlation between TWS estimates and gauged river flow, indicating that data assimilation has considerable potential to downscale GRACE data for hydrological applications. We will also describe how the output from the GRACE land data assimilation system is now being prepared for use in the North American Drought Monitor.

Reichle, Rolf H.↗
NASA NTRS2011

MoSST DAS: The First Working Geomagnetic Data Assimilation System

The Earth possesses an internal magnetic field (geomagnetic field) generated by convection in the outer core (geodynamo). Previous efforts have been focused along two distinct paths: (1) numerical geodynamo modeling to understand the origin of the geomagnetic field, and the mechanisms of geomagnetic secular variations (SV); and (2) geomagnetic field modeling to map the spatial/temporal variations of the field from geomagnetic data, and to derive core properties, e.g. inversion of core flow near the core-mantle boundary (CMB). Geomagnetic data assimilation is a new approach emerged over the past 5 years: surface observations are assimilated with geodynamo models for better understanding of the core dynamical state, and accurately prediction of SV. In collaboration with several geomagnetic research groups, we have developed the first working geomagnetic data assimilation system, Modular, Scalable, Self-consistent, and Three-dimensional (MoSST) DAS, that includes the MoSST numerical dynamo model; 7000 years of geomagnetic field maps from several field models utilizing satellite and ground observatory data, historical magnetic records and archeo/paleo magnetic data; and an ensemble based optimal interpolation (01) assimilation algorithm. With this system, we have demonstrated clearly that the assimilated core dynamical state is substantially different from those of pure geodynamo simulations. Ensemble assimilation runs also show the convergence of the assimilated solutions inside the core, suggesting that the simulation state is pulled closer to the truth via data assimilation. The forecasts from this system are also very accurate: the 5-year forecast of the geomagnetic field agrees very well with the observations; and the 5-year secular variation forecast is more accurate than the IGRF SV forecast models in the past. Using geomagnetic records up to 2009, we have made an SV forecast for the period from 2010-2015, and is a candidate SV model for IGRF-11.

Kuang, Weijia↗
NASA NTRS2006

Efficient Agent-Based Cluster Ensembles

Numerous domains ranging from distributed data acquisition to knowledge reuse need to solve the cluster ensemble problem of combining multiple clusterings into a single unified clustering. Unfortunately current non-agent-based cluster combining methods do not work in a distributed environment, are not robust to corrupted clusterings and require centralized access to all original clusterings. Overcoming these issues will allow cluster ensembles to be used in fundamentally distributed and failure-prone domains such as data acquisition from satellite constellations, in addition to domains demanding confidentiality such as combining clusterings of user profiles. This paper proposes an efficient, distributed, agent-based clustering ensemble method that addresses these issues. In this approach each agent is assigned a small subset of the data and votes on which final cluster its data points should belong to. The final clustering is then evaluated by a global utility, computed in a distributed way. This clustering is also evaluated using an agent-specific utility that is shown to be easier for the agents to maximize. Results show that agents using the agent-specific utility can achieve better performance than traditional non-agent based methods and are effective even when up to 50% of the agents fail.

Agogino, Adrian↗