Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Ensembles”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Improving Surface PM2.5 Forecasts in the United States Using an Ensemble of Chemical Transport Model Outputs: 2. Bias Correction with Satellite Data for Rural Areas

This work serves as the second of a two-part study to improve surface PM2.5 forecasts in the continental U.S. through the integrated use of multi satellite aerosol optical depth (AOD) products (MODIS Terra/Aqua and VIIRS DT/DB), multichemical transport model (CTM) (GEOS-Chem, WRF-Chem, and CMAQ) outputs, and ground observations. In Part I of the study, an ensemble Kalman filter (KF) technique using three CTM outputs and ground observations was developed to correct forecast bias and generate a single best forecast of PM2.5 for next day over non rural areas that have surface PM2.5 measurements in the proximity of 125 km. Here, with AOD data, we extended the bias correction into rural areas where the closest air quality monitoring station is at least 125–300 km away. First, we ensembled all of satellite AOD products to yield the single best AOD. Second, we corrected daily PM2.5 in rural areas from multiple models through the AOD spatial pattern between these areas and non rural areas, referred to as “extended ground truth” or EGT, for the present day. Lastly, we applied the KF technique to reduce the forecast bias for next day using the EGT. Our results find that the ensemble of bias-corrected daily PM2.5 from three CTMs for both today and next day show the best performance. Together, the two-part study develops a multimodel and multi-AOD bias-correction technique that has the potential to improve PM2.5 forecasts in both rural and non rural areas in near real time, and be readily implemented at state levels.

Huanxin Zhang↗

Ensemble Forecasting/Assimilation/Emissions Estimation with WRF-Chem/DART: Accomplishments, Lessons Learned, and Future Plans

Over the past ten years, we have been conducting research on regional ensemble atmospheric composition forecasting/data assimilation/emissions estimation with WRF-Chem/DART. WRF- Chem/DART integrates the Weather Research and Forecasting model (WRF) with online chemistry (WRF-Chem) into the Data Assimilation Research Testbed (DART). DART is an ensemble data assimilation system based on the ensemble adjustment Kalman filter (EAKF) with adaptive inflation, localization (physical and state space), and an optional non-Gaussian formulation of the EAKF. DART includes assimilation of meteorological and limited chemical observations. WRF-Chem/DART extends DART to include assimilation of: MOPITT CO; IASI CO and O3; MODIS AOD; OMI O3, NO2, and SO2; TROPOMI CO, O3, NO2, and SO2, TES CO, CO2 (research mode), O3, NH3, and CH4 (research mode); CrIS CO, O3, NH3, CH4 (research mode), and PAN; SCIAMACHY NO2; GOME2a NO2; MLS O3 and HNO3; and proxy TEMPO O3, and NO2 satellite retrievals as raw retrievals or as ‘compact phase space retrievals’ (CPSRs) for profile retrievals. WRF-Chem/DART also assimilates in situ atmospheric composition measurements and uses the ‘state augmentation method’ for emissions estimation. In our presentation, we will provide an overview of WRF-Chem/DART: • applications and results; • lessons learned from: (i) independent versus joint assimilation; (ii) total/partial column versus profile retrieval assimilation; (iii) joint in situ and retrieval assimilation; (iv) assimilation at grid resolutions ranging from 100 km to 4 km; (v) dynamic emissions estimation; and • future work related to wildfire emissions estimation and intercomparison of CMAQ (online)/JEDI and CMAQ (online and offline)/DART.

WRF-Chem/DART↗

Generation of pareto optimal ensembles of calibrated parameter sets for climate models.

Climate models have a large number of inputs and outputs. In addition, diverse parameters sets can match observations similarly well. These factors make calibrating the models difficult. But as the Earth enters a new climate regime, parameters sets may cease to match observations. History matching is necessary but not sufficient for good predictions. We seek a 'Pareto optimal' ensemble of calibrated parameter sets for the CCSM climate model, in which no individual criteria can be improved without worsening another. One Multi Objective Genetic Algorithm (MOGA) optimization typically requires thousands of simulations but produces an ensemble of Pareto optimal solutions. Our simulation budget of 500-1000 runs allows us to perform the MOGA optimization once, but with far fewer evaluations than normal. We devised an analytic test problem to aid in the selection MOGA settings. The test problem's Pareto set is the surface of a 6 dimensional hypersphere with radius 1 centered at the origin, or rather the portion of it in the [0,1] octant. We also explore starting MOGA from a space-filling Latin Hypercube sample design, specifically Binning Optimal Symmetric Latin Hypercube Sampling (BOSLHS), instead of Monte Carlo (MC). We compare the Pareto sets based on: their number of points, N, larger is better; their RMS distance, d, to the ensemble's center, 0.5553 is optimal; their average radius, {mu}(r), 1 is optimal; their radius standard deviation, {sigma}(r), 0 is optimal. The estimated distributions for these metrics when starting from MC and BOSLHS are shown in Figs. 1 and 2.

54 ENVIRONMENTAL SCIENCES↗

Matlab Cluster Ensemble Toolbox v. 1.0

This is a Matlab toolbox for investigating the application of cluster ensembles to data classification, with the objective of improving the accuracy and/or speed of clustering. The toolbox divides the cluster ensemble problem into four areas, providing functionality for each. These include, (1) synthetic data generation, (2) clustering to generate individual data partitions and similarity matrices, (3) consensus function generation and final clustering to generate ensemble data partitioning, and (4) implementation of accuracy metrics. With regard to data generation, Gaussian data of arbitrary dimension can be generated. The kcenters algorithm can then be used to generate individual data partitions by either, (a) subsampling the data and clustering each subsample, or by (b) randomly initializing the algorithm and generating a clustering for each initialization. In either case an overall similarity matrix can be computed using a consensus function operating on the individual similarity matrices. A final clustering can be performed and performance metrics are provided for evaluation purposes.

Source record↗

Ensemble Solar Forecasting Statistical Quantification and Sensitivity Analysis: Preprint

Uncertainties associated with solar forecasts present challenges to maintain grid reliability, especially at high solar penetrations. This study aims to quantify the errors associated with the day-ahead solar forecast parameters and the theoretical solar power output for a 51-kW solar power plant in a utility area in the state of Vermont, U.S. Forecasts were generated by three numerical weather prediction (NWP) models, including the Rapid Refresh, the High Resolution Rapid Refresh, and the North American Model, and a machine-learning ensemble model. A photovoltaic (PV) performance model was adopted to calculate theoretical solar power generation using the forecast parameters (e.g., irradiance, cell temperature, and wind speed). Errors of the power outputs were quantified using statistical moments and a suite of metrics, such as the normalized root mean squared error (NRMSE). In addition, the PV model's sensitivity to different forecast parameters was quantified and analyzed. Results showed that the ensemble model yielded forecasts in all parameters with the smallest NRMSE. The NRMSE of solar irradiance forecasts of the ensemble NWP model was reduced by 28.10% compared to the best of the three NWP models. Further, the sensitivity analysis indicated that the errors of the forecasted cell temperature attributed only approximately 0.12% to the NRMSE of the power output as opposed to 7.44% from the forecasted solar irradiance.

14 SOLAR ENERGY↗

Ensemble Solar Forecasting Statistical Quantification and Sensitivity Analysis

Uncertainties associated with solar forecasts present challenges to maintain grid reliability, especially at high solar penetrations. This study aims to quantify the errors associated with the day-ahead solar forecast parameters and the theoretical solar power output for a 51-kW solar power plant in a utility area in the state of Vermont, U.S. Forecasts were generated by three numerical weather prediction (NWP) models, including the Rapid Refresh, the High Resolution Rapid Refresh, and the North American Model, and a machine-learning ensemble model. A photovoltaic (PV) performance model was adopted to calculate theoretical solar power generation using the forecast parameters (e.g., irradiance, cell temperature, and wind speed). Errors of the power outputs were quantified using statistical moments and a suite of metrics, such as the normalized root mean squared error (NRMSE). In addition, the PV model's sensitivity to different forecast parameters was quantified and analyzed. Results showed that the ensemble model yielded forecasts in all parameters with the smallest NRMSE. The NRMSE of solar irradiance forecasts of the ensemble NWP model was reduced by 28.10% compared to the best of the three NWP models. Further, the sensitivity analysis indicated that the errors of the forecasted cell temperature attributed only approximately 0.12% to the NRMSE of the power output as opposed to 7.44% from the forecasted solar irradiance.

14 SOLAR ENERGY↗

Gradient Flow Analysis on MILC HISQ Ensembles

We report on a preliminary scale determination with gradient-flow techniques on the $N_f = 2 + 1 + 1$ HISQ ensembles generated by the MILC collaboration. The ensembles include four lattice spacings, ranging from 0.15 to 0.06 fm, and both physical and unphysical values of the quark masses. The scales $\sqrt{t_0}/a$ and $w_0/a$ are computed using Symanzik flow and the cloverleaf definition of $\langle E \rangle$ on each ensemble. Then both scales and the meson masses $aM_\pi$ and $aM_K$ are adjusted for mistunings in the charm mass. Using a combination of continuum chiral perturbation theory and a Taylor series ansatz in the lattice spacing, the results are simultaneously extrapolated to the continuum and interpolated to physical quark masses. Our preliminary results are $\sqrt{t_0} = 0.1422(7)$fm and $w_0 = 0.1732(10)$fm. We also find the continuum mass-dependence of $w_0$.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Exploring flexible communications for streamlining DNN ensemble training pipelines

Parallel training of a Deep Neural Network (DNN) ensemble on a cluster of nodes is a common practice to train multiple models in order to construct a model with a higher prediction accuracy, or to quickly tune the parameters of a training model. Existing ensemble training pipelines perform a great deal of redundant operations, resulting in unnecessary CPU usage, or even poor pipeline performance. In order to remove these redundancies, we need pipelines with more communication flexibility than existing DNN frameworks can provide. This project investigates a series of designs to improve pipeline flexibility and adaptivity, while also increasing performance. We implement our designs using Tensorflow with Horovod, and test it using several large DNNs in a large scale GPU cluster, the Titan supercomputer at Oak Ridge National Lab. Our results show that with the new flexible communication schemes, the CPU time spent during training is reduced by 2-11X. Furthermore, our implementation can achieve up to 10X speedups when CPU core limits are imposed. Our best pipeline also reduces the average power draw of the ensemble training process by 5--16% when compared to the baseline.

Pittman, Randall↗

Situation awareness and dynamic ensemble forecasting of abnormal behavior in cyber-physical system

A plurality of monitoring nodes may each generate a time-series of current monitoring node values representing current operation of a cyber-physical system. A feature-based forecasting framework may receive the time-series of and generate a set of current feature vectors using feature discovery techniques. The feature behavior for each monitoring node may be characterized in the form of decision boundaries that separate normal and abnormal space based on operating data of the system. A set of ensemble state-space models may be constructed to represent feature evolution in the time-domain, wherein the forecasted outputs from the set of ensemble state-space models comprise anticipated time evolution of features. The framework may then obtain an overall features forecast through dynamic ensemble averaging and compare the overall features forecast to a threshold to generate an estimate associated with at least one feature vector crossing an associated decision boundary.

Abbaszadeh, Masoud↗

A generalized polynomial chaos based ensemble Kalman filter with high accuracy

As one of the most adopted sequential data assimilation methods in many areas, especially those involving complex nonlinear dynamics, the ensemble Kalman filter (EnKF) has been under extensive investigation regarding its properties and efficiency. Compared to other variants of the Kalman filter (KF), EnKF is straightforward to implement, as it employs random ensembles to represent solution states. This, however, introduces sampling errors that affect the accuracy of EnKF in a negative manner. Though sampling errors can be easily reduced by using a large number of samples, in practice this is undesirable as each ensemble member is a solution of the system of state equations and can be time consuming to compute for large-scale problems. In this paper we present an efficient EnKF implementation via generalized polynomial chaos (gPC) expansion. The key ingredients of the proposed approach involve (1) solving the system of stochastic state equations via the gPC methodology to gain efficiency; and (2) sampling the gPC approximation of the stochastic solution with an arbitrarily large number of samples, at virtually no additional computational cost, to drastically reduce the sampling errors. The resulting algorithm thus achieves a high accuracy at reduced computational cost, compared to the classical implementations of EnKF. Numerical examples are provided to verify the convergence property and accuracy improvement of the new algorithm. We also prove that for linear systems with Gaussian noise, the first-order gPC Kalman filter method is equivalent to the exact Kalman filter.

97 MATHEMATICS AND COMPUTING↗

Uncertainty based Online Ensemble on Non-Stationary Data for Fusion Science

Machine Learning (ML) is poised to play a pivotal role in the development and operation of next-generation fusion devices. Fusion data shows non-stationary behavior due to drifts in the data. The drifts can arise from both experimental evolution and machine wear-and-tear. ML models assume stationary distribution and fail to maintain performance when encountered with non-stationary data streams.Online learning can be used to continuously adapt the models with new data as it is acquired. However, traditional online learning can suffer from short-term performance degradation, as ground truth are not available before making the prediction. To address this challenge, we propose uncertainty aware ensemble approach for online learning. We use Deep Gaussian Process Approximation (DGPA) technique for calibrated uncertainty estimation and use the uncertainty values to guide a meta-algorithm that produces predictions based on ensemble of learners. Moreover, DGPA also provides uncertainty estimation along with the predictions for decision makers. This paper demonstrates that the proposed method outperforms traditional online learning approach, and a naive ensemble without uncertainty guidance by about 7% and 6%, respectively, on B-coil deflection prediction at DIII-D Fusion Facility.

Rajput, Kishansingh [Thomas Jefferson National Acc↗

A Flexible Approach for the Statistical Visualization of Ensemble Data

Scientists are increasingly moving towards ensemble data sets to explore relationships present in dynamic systems. Ensemble data sets combine spatio-temporal simulation results generated using multiple numerical models, sampled input conditions and perturbed parameters. While ensemble data sets are a powerful tool for mitigating uncertainty, they pose significant visualization and analysis challenges due to their complexity. We present a collection of overview and statistical displays linked through a high level of interactivity to provide a framework for gaining key scientific insight into the distribution of the simulation results as well as the uncertainty associated with the data. In contrast to methods that present large amounts of diverse information in a single display, we argue that combining multiple linked statistical displays yields a clearer presentation of the data and facilitates a greater level of visual data analysis. We demonstrate this approach using driving problems from climate modeling and meteorology and discuss generalizations to other fields.

99 GENERAL AND MISCELLANEOUS↗

Perturbed Physics Ensemble Simulations of Cirrus on the Cloud System-resolving Scale

In this study, the effect of uncertainties in the parameterization of ice microphysical processes and initial conditions on the variability of cirrus microphysical and radiative properties are investigated in a series of cloud system-resolving perturbed physics ensemble (PPE) and initial condition ensemble (ICE) simulations. Three cirrus cases representative of mid-latitude, subtropical and tropical cirrus are examined. It is found that the variability in cirrus properties induced by perturbing uncertain parameters in ice microphysics parameterizations outweighs the variability induced by perturbing the initial conditions in midlatitude and subtropical cirrus. However, in tropical anvil cirrus the variability in the PPE and ICE simulations is about the same order of magnitude. The cirrus properties showing the largest sensitivity are ice water content (IWC) and cloud thickness whereas the averaged high cloud cover is only marginally affected. Changes in cirrus ice water path and outgoing longwave radiation are controlled primarily by changes in IWC and cloud thickness but not by changes is the averaged high cloud cover. The change in the vertical distribution of cloud fraction and cloud thickness is caused by changes in cirrus cloud base whereas cloud top is not sensitive to either perturbed physics or perturbed initial conditions. In all cirrus cases, the top three parameters controlling the microphysical variability and radiative impact of cirrus clouds are ice fall speeds, ice autoconversion size thresholds and heterogeneous ice nucleation. Changes in the ice deposition coefficient do not affect the ice water path and outgoing longwave radiation. Similarly, changes in the number concentration of aerosols available for homogeneous freezing have virtually no effect on the microphysical and radiative properties of midlatitude and subtropical cirrus but only little impact on tropical anvil cirrus. Overall, the sensitivity of cirrus microphysical and radiative properties to uncertainties in ice microphysics is largest for midlatitude cirrus and smallest for tropical anvil cirrus.

cirrus↗

Enhanced accuracy through ensembling of randomly initialized auto-regressive models for dynamical systems

Computational mechanics simulations using traditional finite element methods (FEM) require prohibitively expensive computational resources for real-time engineering applications, design optimization, and digital twin implementations. While machine learning (ML) surrogate models offer significant computational speedups, autoregressive ML models for time-dependent mechanical systems suffer from error accumulation that compromises long-term prediction reliability - a critical concern for engineering applications where accuracy over extended time horizons is essential for safety and performance assessments. Here, we propose a deep ensemble framework specifically designed to address this challenge in computational mechanics applications, where multiple ML surrogate models with random weight initializations are trained in parallel and their predictions aggregated during inference. This approach leverages statistical diversity to maximize information gain from a fixed set of training data and to mitigate error propagation, while maintaining the computational efficiency that makes ML surrogates attractive for engineering practice. We validate the framework on three representative problems spanning critical areas of computational mechanics: stress field evolution in heterogeneous microstructures under complex loading (relevant to advanced materials design and composite analysis), planetary-scale shallow water dynamics (applicable to environmental and geotechnical engineering), and Gray-Scott reaction-diffusion systems (relevant to mass transport and chemical process engineering). Across all test cases, the ensemble approach demonstrates consistent error reduction of 15-33% compared to individual models. The codes for this work are available on GitHub (https://github.com/Graham-Brady-Research-Group/AutoregressiveEnsemble_SpatioTemporal_Evolution).

autoregressive prediction↗

Developing a heterogeneous ensemble learning framework to evaluate Alkali-silica reaction damage in concrete using acoustic emission signals

The monitoring and evaluation of Alkali-silica reaction (ASR) damage in concrete structures are required to ensure the serviceability and integrity of concrete infrastructures such as bridges and dams. The innovation of this paper lies in the development of an automatic ASR monitoring and evaluation approach by leveraging acoustic emission (AE) and a heterogeneous ensemble learning framework. Here, in this paper, ASR was monitored by AE sensors attached to a concrete specimen, which was placed in a chamber with high humidity and temperature. The recorded AE signals were filtered and divided by four ASR phases according to signal strength, crack width and expansion strain. A heterogeneous ensemble network including convolutional neural networks (CNN) and random forest models was employed to learn different features from AE signals and classify the AE signals into their corresponding phases. The results suggest that the proposed model has a high performance and classifies the signals into the assigned phases with high accuracy.

42 ENGINEERING↗

Parametric Sensitivity and Uncertainty Quantification in the Version 1 of E3SM Atmosphere Model Based on Short Perturbed Parameter Ensemble Simulations

Abstract The atmospheric component of Energy Exascale Earth System Model version 1 has included many new features in the physics parameterizations compared to its predecessors. Potential complex nonlinear interactions among the new features create a significant challenge for understanding the model behaviors and parameter tuning. Using the one‐at‐a‐time method, the benefit of tuning one parameter may offset the benefit of tuning another parameter, or improvement in one target variable may lead to degradation in another target variable. To better understand the Energy Exascale Earth System Model version 1 model behaviors and physics, we conducted a large number of short simulations (three days) in which 18 parameters carefully selected from parameterizations of deep convection, shallow convection, and cloud macrophysics and microphysics were perturbed simultaneously using the Latin hypercube sampling method. From the perturbed parameter ensemble simulations and use of different skill score functions, we identified the most sensitive parameters, quantified how the model responds to changes of the parameters for both global mean and spatial distribution, and estimated the maximum likelihood of model parameter space for a number of important fidelity metrics. Comparison of the parametric sensitivity using simulations of two different lengths suggests that perturbed parameter ensemble using short simulations has some bearing on understanding parametric sensitivity of longer simulations. Results from this analysis provide a more comprehensive picture of the Energy Exascale Earth System Model version 1 behavior. The difficulty in reducing biases in multiple variables simultaneously highlights the need of characterizing model structural uncertainty (so‐called embedded errors) to inform future development efforts.

54 ENVIRONMENTAL SCIENCES↗

Estimating Ocean Observation Impacts on Coupled Atmosphere‐Ocean Models Using Ensemble Forecast Sensitivity to Observation (EFSO)

Ensemble Forecast Sensitivity to Observation (EFSO) is a technique that can efficiently identify the beneficial/detrimental impacts of every observation in ensemble-based data assimilation (DA). While EFSO has been successfully employed on atmospheric DA, it has never been applied to ocean or coupled DA due to the lack of a suitable error norm for oceanic variables. This study introduces a new density-based error norm incorporating sea temperature and salinity forecast errors, making EFSO applicable to ocean DA for the first time. We implemented the oceanic EFSO on the CFSv2-LETKF and investigated the impact of ocean observations under a weakly coupled DA framework. By removing the detrimental ocean observations detected by EFSO, the CFSv2 forecasts were significantly improved, showing the validation of impact estimation and the great potential of EFSO to be extended as a data selection criterion.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States

This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.

60 APPLIED LIFE SCIENCES↗