Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model diagnostics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

Decomposition of the Mean Squared Error and NSE Performance Criteria: Implications for Improving Hydrological Modelling

The mean squared error (MSE) and the related normalization, the Nash-Sutcliffe efficiency (NSE), are the two criteria most widely used for calibration and evaluation of hydrological models with observed data. Here, we present a diagnostically interesting decomposition of NSE (and hence MSE), which facilitates analysis of the relative importance of its different components in the context of hydrological modelling, and show how model calibration problems can arise due to interactions among these components. The analysis is illustrated by calibrating a simple conceptual precipitation-runoff model to daily data for a number of Austrian basins having a broad range of hydro-meteorological characteristics. Evaluation of the results clearly demonstrates the problems that can be associated with any calibration based on the NSE (or MSE) criterion. While we propose and test an alternative criterion that can help to reduce model calibration problems, the primary purpose of this study is not to present an improved measure of model performance. Instead, we seek to show that there are systematic problems inherent with any optimization based on formulations related to the MSE. The analysis and results have implications to the manner in which we calibrate and evaluate environmental models; we discuss these and suggest possible ways forward that may move us towards an improved and diagnostically meaningful approach to model performance evaluation and identification.

Gupta, Hoshin V.↗
NASA NTRS1992

A Lagrangian analysis of a sudden stratospheric warming - Comparison of a model simulation and LIMS observations

Lagrangian area diagnostics and trajectory techniques are used to investigate the radiative and dynamical characteristics of a spontaneous sudden warming which occurred during a 2-yr Langley Research Center model simulation. The ability of the Langley Research Center GCM to simulate the major features of the stratospheric circulation during such highly disturbed periods is illustrated by comparison of the simulated warming to the observed circulation during the LIMS observation period. The apparent sink of vortex area associated with Rossby wave-breaking accounts for the majority of the reduction of the size of the vortex and also acts to offset the radiatively driven increase in the area occupied by the 'surf zone'. Trajectory analysis of selected material lines substantiates the conclusions from the area diagnostics.

Pierce, R. B.↗
DOE OSTI2026

Avoidance of disruptions on KSTAR due to vertical displacement events via novel real-time stability assessment

Disruption avoidance via the DECAF approach has been achieved on KSTAR using a novel real-time vertical stability assessment and a multiactuator feedback control strategy. The development of disruption avoidance strategies with reactor-relevant reliability is an urgent activity, enabling future fusion power plants. The stability metric employed is based on a new formulation of a vertical force gradient balance metric evaluated across the poloidal cross section of the plasma, with parameters tuned using historical data. Evaluation of this metric on a validation set of 400 recent KSTAR shots indicates >82% of Vertical displacement events can be avoided via feedback control. Essential to its calculation is the two-dimensional toroidal current density distribution in the plasma. Measurement of this profile faster than fully-converged equilibrium reconstructions can deliver is found to improve forecaster performance and is achieved with a surrogate model that takes as input magnetic diagnostic measurements and outputs the current profile on a basis comprising the top principal components of historical current profiles (from past equilibrium reconstructions). This method solves the non-uniqueness problem typically faced when reconstructing current profiles directly from diagnostics, while improving computational time and accuracy. On average, profiles produced by this model reach coefficients of determination of >0.99 with respect to those from equilibrium reconstructions. The avoidance actuators employed include poloidal field coils and an electron cyclotron current drive system. The multiactuator approach, as shown in this first demonstration, allows disruption avoidance while minimizing impact to operational performance. This ability, along with its flexibility and speed, makes this new approach an attractive option for avoiding these types of disruptions in reactors.

Tobin, Matthew [Columbia Univ., New York, NY (Unit↗
NASA NTRS2024

Interpretable Machine Learning for Molecular Biosignatures: a Novel Single-Sample Feature Importance Method That Is Sensitive To Statistical Interactions

Isotope ratio mass spectrometry (IRMS) of volatiles (e.g., CO 2 ) promises to be a powerful tool for potential biosignature detection for future missions to ocean worlds (OW) such as Europa and Enceladus. Machine learning (ML) methods for IRMS data could enable science autonomy by onboard prediction of seawater chemistry and biosignature presence. However, ML models are likely to be complex and involve statistical interactions between features (variables), which can make predictions seem opaque and enigmatic. For ML predictions as significant as extraterrestrial biosignatures, we must place extraordinary confidence in models. It is therefore essential that these models make interpretable predictions (i.e., human-understandable) and include false-prediction diagnostics. We achieve high accuracy and interpretability in ML biosignature and seawater chemistry models for OW through a nearest-neighbors feature selection tool that detects statistical interactions between predictors, constructs interaction networks for visualization of selected features working together to make a prediction, and reports single-sample feature importance scores for false-detection diagnostics. Here we develop a novel single-sample nearest-neighbors projected distance regression(ssNPDR) feature selection method that improves upon existing single-sample algorithms through the inclusion of statistical interactions while providing false-prediction diagnostics for ML models.

geochemistry↗
DOE OSTI2014

A new fast two-color interferometer at Alcator C-Mod for turbulence measurements and comparison with phase contrast imaging

Accurately predicting the turbulent transport properties of magnetically confined plasmas is a major challenge of fusion energy research. Validation of transport models is typically done by applying so-called “synthetic diagnostics” to the output of nonlinear gyrokinetic simulations, and the results are compared to experimental data. As part of the validation process, comparing two independent turbulence measurements to each other provides the opportunity to test the synthetic diagnostics themselves; a step which is rarely possible due to limited availability of redundant fluctuation measurements on magnetic confinement experiments. At Alcator C-Mod, phase-contrast imaging (PCI) is a commonly used turbulence diagnostic. PCI measures line-integrated electron density fluctuations with high sensitivity and wavenumber resolution (1.6 cm{sup −1}≲|k{sub R}|≲11 cm{sup −1}). A new fast two-color interferometry (FTCI) diagnostic on the Alcator C-Mod tokamak measures long-wavelength (|k{sub R}|≲3.0 cm{sup −1}) line-integrated electron density fluctuations. Measurements of coherent and broadband fluctuations made by PCI and FTCI are compared here for the first time. Good quantitative agreement is found between the two measurements. This provides experimental validation of the low-wavenumber region of the PCI calibration, and also helps validate the low-wavenumber portions of the synthetic PCI diagnostic that has been used in gyrokinetic model validation work in the past. We discuss possibilities to upgrade FTCI, so that a similar comparison could be done at higher wavenumbers in the future.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2012

Application of the CALIOP Layer Product to Evaluate the Vertical Distribution of Aerosols Estimated by Global Models: AeroCom Phase I Results

The CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) layer product is used for a multimodel evaluation of the vertical distribution of aerosols. Annual and seasonal aerosol extinction profiles are analyzed over 13 sub-continental regions representative of industrial, dust, and biomass burning pollution, from CALIOP 2007-2009 observations and from AeroCom (Aerosol Comparisons between Observations and Models) 2000 simulations. An extinction mean height diagnostic (Z-alpha) is defined to quantitatively assess the models' performance. It is calculated over the 0-6 km and 0-10 km altitude ranges by weighting the altitude of each 100 m altitude layer by its aerosol extinction coefficient. The mean extinction profiles derived from CALIOP layer products provide consistent regional and seasonal specificities and a low inter-annual variability. While the outputs from most models are significantly correlated with the observed Z-alpha climatologies, some do better than others, and 2 of the 12 models perform particularly well in all seasons. Over industrial and maritime regions, most models show higher Z-alpha than observed by CALIOP, whereas over the African and Chinese dust source regions, Z-alpha is underestimated during Northern Hemisphere Spring and Summer. The positive model bias in Z-alpha is mainly due to an overestimate of the extinction above 6 km. Potential CALIOP and model limitations, and methodological factors that might contribute to the differences are discussed.

Aerosols and particles↗
NASA NTRS2005

Lessons Learned in the Livingstone 2 on Earth Observing One Flight Experiment

The Livingstone 2 (L2) model-based diagnosis software is a reusable diagnostic tool for monitoring complex systems. In 2004, L2 was integrated with the JPL Autonomous Sciencecraft Experiment (ASE) and deployed on-board Goddard's Earth Observing One (EO-1) remote sensing satellite, to monitor and diagnose the EO-1 space science instruments and imaging sequence. This paper reports on lessons learned from this flight experiment. The goals for this experiment, including validation of minimum success criteria and of a series of diagnostic scenarios, have all been successfully net. Long-term operations in space are on-going, as a test of the maturity of the system, with L2 performance remaining flawless. L2 has demonstrated the ability to track the state of the system during nominal operations, detect simulated abnormalities in operations and isolate failures to their root cause fault. Specific advances demonstrated include diagnosis of ambiguity groups rather than a single fault candidate; hypothesis revision given new sensor evidence about the state of the system; and the capability to check for faults in a dynamic system without having to wait until the system is quiescent. The major benefits of this advanced health management technology are to increase mission duration and reliability through intelligent fault protection, and robust autonomous operations with reduced dependency on supervisory operations from Earth. The work-load for operators will be reduced by telemetry of processed state-of-health information rather than raw data. The long-term vision is that of making diagnosis available to the onboard planner or executive, allowing autonomy software to re-plan in order to work around known component failures. For a system that is expected to evolve substantially over its lifetime, as for the International Space Station, the model-based approach has definite advantages over rule-based expert systems and limit-checking fault protection systems, as these do not scale well. The model-based approach facilitates reuse of the L2 diagnostic software; only the model of the system to be diagnosed and telemetry monitoring software has to be rebuilt for a new system or expanded for a growing system. The hierarchical L2 model supports modularity and expendability, and as such is suitable solution for integrated system health management as envisioned for systems-of-systems.

Hayden, Sandra C.↗
DOE OSTI2021

A Modified Vegetation Photosynthesis and Respiration Model (VPRM) for the Eastern USA and Canada, Evaluated With Comparison to Atmospheric Observations and Other Biospheric Models

Atmospheric CO 2 measurements from a dense surface network can help to evaluate terrestrial biosphere model (TBM) simulations of Net Ecosystem Exchange (NEE) with two key benefits. First, gridded CO 2 flux estimates can be evaluated over regional scales, not possible using flux tower observations at discrete locations for model evaluation. Second, TBM ability to explain atmospheric CO 2 fluctuations due to the biosphere can be directly tested, an important objective for anthropogenic emissions monitoring using atmospheric observations. Here, we customize the Vegetation Photosynthesis and Respiration Model (VPRM) for an eastern North American domain with strong biological activity upwind of urban areas. Parameters are optimized using flux tower observations from a historical database with sites in (and near) the domain. In addition, the respiration model (originally a linear function of temperature) is modified to account for impacts of changing foliage, non-linear temperature, and water stress. Flux estimates from VPRM, the Carnegie-Ames-Stanford Approach (CASA) model and the Simple Biosphere Model v4 (SiB4), are convolved with footprints from atmospheric transport models for evaluation with CO 2 observations at 21 towers in the domain, with roughly half of the towers used here for the first time. Results show that the new respiration model in VPRM helps to correct a growing season sink bias in the atmosphere associated with underestimated summertime respiration using the original model with annual parameters. The new VPRM also better explains fine-scale atmospheric CO 2 variability compared to other TBMs, due to higher resolution diagnostic phenology, the new respiration model, domain-specific parameters, and high-quality input data sets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A laser powder bed fusion system for operando synchrotron x-ray imaging and correlative diagnostic experiments at the Stanford Synchrotron Radiation Lightsource

Laser powder bed fusion (LPBF) is a highly dynamic multi-physics process used for the additive manufacturing (AM) of metal components. Improving process understanding and validating predictive computational models require high-fidelity diagnostics capable of capturing data in challenging environments. Synchrotron x-ray techniques play a vital role in the validation process as they are the only in situ diagnostic capable of imaging sub-surface melt pool dynamics and microstructure evolution during LPBF-AM. In this article, a laboratory scale system designed to mimic LPBF process conditions while operating at a synchrotron facility is described. The system is implemented with process accurate atmospheric conditions, including an air knife for active vapor plume removal. Significantly, the chamber also incorporates a diagnostic sensor suite that monitors emitted optical, acoustic, and electronic signals during laser processing with coincident x-ray imaging. The addition of the sensor suite enables validation of these industrially compatible single point sensors by detecting pore formation and spatter events and directly correlating the events with changes in the detected signal. Experiments in the Ti–6Al–4V alloy performed at the Stanford Synchrotron Radiation Lightsource using the system are detailed with sufficient sampling rates to probe melt pool dynamics. X-ray imaging captures melt pool dynamics at frame rates of 20 kHz with a 2 µm pixel resolution, and the coincident diagnostic sensor data are recorded at 470 kHz. Furthermore, this work shows that the current system enables the in situ detection of defects during the LPBF process and permits direct correlation of diagnostic signatures at the exact time of defect formation.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Neural Posterior Estimation for Scalable and Accurate Inverse Parameter Inference in Li-Ion Batteries

Diagnosing the internal state of Li-ion batteries is critical for battery research, operation of real-world systems, and prognostic evaluation of remaining lifetime. By using physics-based models to perform probabilistic parameter estimation via Bayesian calibration, diagnostics can account for the uncertainty due to model fitness, data noise, and the observability of any given parameter. However, Bayesian calibration in Li-ion batteries using electrochemical data is computationally intensive even when using a fast surrogate in place of physics-based models, requiring many thousands of model evaluations. A fully amortized alternative is neural posterior estimation (NPE). NPE shifts the computational burden from the parameter estimation step to data generation and model training, reducing the parameter estimation time from minutes to milliseconds, enabling real-time applications. The present work shows that NPE can infer parameters equally or more accurately than Bayesian calibration, even if it leads to higher voltage reconstruction errors. We also demonstrate that the higher computational costs for data generation are tractable even in high-dimensional cases (ranging from 6 to 27 estimated parameters). The NPE method also offers several interpretability advantages over Bayesian calibration, such as local parameter sensitivity to specific regions of the voltage curve. The NPE method is demonstrated using an experimental fast charge dataset, with parameter estimates validated against measurements of loss of lithium inventory and loss of active material. The implementation is made available in a companion repository (https://github.com/NatLabRockies/BatFIT).

25 ENERGY STORAGE↗
DOE OSTI2022

Method to account for natural fracture induced elastic anisotropy in geomechanical characterization of shale gas reservoirs

Shale has been usually recognized as a transverse isotropic (TI) medium in conventional geomechanical log interpretation due to its laminated nature. However, when natural fractures exist in the shale rock, additional elastic anisotropy is introduced, converting laminated Shale to an orthorhombic (OB) medium. Previous studies illustrate that neglecting the natural fracture induced anisotropy in shale geomechanical log interpretation could lead to inaccurate evaluations of elastic moduli and in-situ stresses. In this paper, a new method is developed to account for the natural fracture induced anisotropy in geomechanical log interpretation based upon the TI acoustic model developed by the author and a characterization technique of elastic wave anisotropy (Sayers, 1991). The new OB model incorporates the four acoustic log data inputs and five modeling constraints in a nonlinear optimization algorithm to solve for the nine independent stiffness coefficients of an OB rock, and further to solve for the geomechanical properties and in-situ stress profiles in an OB formation. The new method was validated with a Marcellus Gas Shale field case. Both the new OB model and the conventional TI model were applied to interpret the minimum horizontal stress profile for the same formation. By comparing the results, the OB model is more robust than the TI from two aspects. First, the average stress magnitude predicted by the OB model is closer to the one measured by the Diagnostic Fracture Injection Test (DFIT). Second, the OB model predicts a more obvious stress barrier between the lower Marcellus and upper Onondaga Limestone than the TI model does. Finally, the predicted stress barrier is consistent with the observation of the microseismic events of a horizontal well drilled and completed nearby, which reveals that no hydraulic fracture propagates downward through the bottom boundary of Marcellus Shale into the underlying Onondaga Limestone.

03 NATURAL GAS↗
DOE OSTI2023

Vertical control of DIII-D discharges with strong negative triangularity

Through predictive modeling validated by a series of experiments on DIII-D, the vertical stability of low β diverted plasmas with strong negative triangularity (NT) ($\delta\sim-0.6$) is assessed. As a result of their unique magnetic geometry, NT plasmas feature larger Shafranov shifts and more elongated inner flux surfaces than positive triangularity counterparts, typically leading to enhanced vertical instability growth rates. However, coupling with the non-conformal vessel DIII-D wall reduces these growth rates to controllable values, providing a path forward for stabilizing strongly NT plasma with a diverted geometry. These discharges are used to validate GSdesign (part of the TokSys code suite) stability calculations in strong NT plasmas on DIII-D, with errors of no more than ~20% observed between modeled and experimentally measured growth rates. Additions of diagnostic noise and power supply tuning to the TokSys model are needed to accurately capture the time dependence of DIII-D NT discharges, assisting with the design of control schemes specific to the DIII-D poloidal field coils. Lastly, implications of these results on a future NT reactor are briefly described.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Deep-learning-driven simulations of boundary layer clouds over the Southern Great Plains

Abstract. Based on long-term observations at the Southern Great Plains site by the Atmospheric Radiation Measurement (ARM) program for training and validation, a deep-learning model is developed to simulate the daytime evolution of boundary layer clouds (BLCs) from the perspective of land–atmosphere coupling. The model takes ARM measurements (including early-morning soundings and diurnally varying surface meteorological conditions and heat fluxes) as inputs and predicts hourly estimates (including cloud occurrence, the positions of cloud boundaries, and the vertical profile of the cloud fraction) as outputs. The deep-learning model offers good agreement with the observed cloud fields, especially in the accuracy with which cloud occurrence and base height are reproduced. When the inputs are substituted by reanalysis data from ERA5 and MERRA-2, the outputs of the deep-learning model provide a better agreement with observation than the cloud fields extracted from ERA5 and MERRA-2 themselves. Thus, the deep-learning model shows great potential to serve as a diagnostic tool for the performance of physics-based models in simulating stratiform and cumulus clouds. By quantifying biases in clouds and attributing them to the simulated atmospheric state variables versus the model-parameterized cloud processes, this observation-based deep-learning model may offer insights into the directions needed to improve the simulation of BLCs in physics-based models for weather forecasting and climate prediction.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1989

CONFIG - Adapting qualitative modeling and discrete event simulation for design of fault management systems

CONFIG is a modeling and simulation tool prototype for analyzing the normal and faulty qualitative behaviors of engineered systems. Qualitative modeling and discrete-event simulation have been adapted and integrated, to support early development, during system design, of software and procedures for management of failures, especially in diagnostic expert systems. Qualitative component models are defined in terms of normal and faulty modes and processes, which are defined by invocation statements and effect statements with time delays. System models are constructed graphically by using instances of components and relations from object-oriented hierarchical model libraries. Extension and reuse of CONFIG models and analysis capabilities in hybrid rule- and model-based expert fault-management support systems are discussed.

Malin, Jane T.↗
DOE OSTI2014

Detailed analysis of hollow ions spectra from dense matter pumped by X-ray emission of relativistic laser plasma

X-ray emission from hollow ions offers new diagnostic opportunities for dense, strongly coupled plasma. We present extended modeling of the x-ray emission spectrum reported by Colgan et al. [Phys. Rev. Lett. 110, 125001 (2013)] based on two collisional-radiative codes: the hybrid-structure Spectroscopic Collisional-Radiative Atomic Model (SCRAM) and the mixed-unresolved transition arrays (MUTA) ATOMIC model. We show that both accuracy and completeness in the modeled energy level structure are critical for reliable diagnostics, investigate how emission changes with different treatments of ionization potential depression, and discuss two approaches to handling the extensive structure required for hollow-ion models with many multiply excited configurations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1987

Detectability of minerals on desert alluvial fans using reflectance spectra

The visible and near-infrared reflectance spectra of soil samples collected from desert alluvial and colluvial surfaces in the Cuprite mining district, Nevada, were analyzed. These surfaces are downslope from hydrothermally altered volcanic rocks that contain spectrally characteristic minerals such as alunite and kaolinite. Coarse fractions of the soils on the alluvial fans are mineralogically variable and express the upslope lithologies; fine fractions are remarkably similar mineralogically and spectrally in all samples because of dilution of local mineral components by regionally derived windblown dust. Theoretical models for spectral mixing and for particle-size effects were used to model the observed spectral variations. Diagnostic mineral absorption bands in the spectra of fan materials were enhanced by computationally removing the spectrum of the homogeneous fine-soil component. Results show that spectral mixing models are useful for analyzing data with high spectral resolution obtained by field and aircraft spectrometers.

Shipman, Hugh↗