SEARCH · Engineering Papers
Results for “error analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
MS25: Materials Science-Focused Benchmark Data Set for Machine Learning Interatomic Potentials
Here, we present MS25, a benchmark data set for evaluating machine learning interatomic potentials (MLIPs) across diverse materials-relevant systems including MgO surfaces, liquid water, zeolites, a catalytic Pt surface reaction, high-entropy alloys (HEAs), and disordered Zr-oxides. Five MLIP architectures (MACE, NequIP, Allegro, MTP, and Torch-ANI) are trained and tested, focusing not only on traditional metrics (energies, forces, and stresses) but also explicitly validating derived physical observables such as lattice constants, volumes, and reaction barriers. We find that most models reach comparable accuracy on standard error metrics across the simple systems, although equivariant MLIPs offer 1.5–2× improvements over nonequivariant MLIPs in energy and force error for structurally complex or compositionally disordered environments such as HEAs and Zr–O systems. Our analysis highlights that low errors in energy and force predictions do not guarantee reliable observables, emphasizing the necessity of explicit validation. We demonstrate limitations in cross-framework transferability, as models trained on one zeolite framework (CHA) fail to reliably generalize to predictions of structurally distinct frameworks (e.g., MFI). Size-extensive tests show some dependence on system size for MgO, resulting from forced periodicity. The HEA and Zr–O data sets are identified as challenging tests for future benchmarks and MLIP model architecture developments as they show significant differentiation in error between MLIP architectures and are still relatively difficult at 1000 training images. Moving forward, we recommend that benchmarking efforts shift their focus from marginal accuracy improvements in energy and force errors toward identifying and understanding model failure modes, rigorously assessing transferability, and evaluating how their errors affect observable predictions. For researchers looking to choose an MLIP architecture, we suggest selecting equivariant MLIP architectures if the complexity of the system is a challenge. For simple materials problems, auxiliary features such as integration with molecular dynamics engines, trade-offs between computational data set generation cost vs MLIP inference speed, and framework integration may play a more important decision factor than small differences in error metrics that are unlikely to matter for production-level research.
Behavior and mechanisms of Doppler wind lidar error in varying stability regimes
Abstract. Wind lidars are widespread and important tools in atmospheric observations. An intrinsic part of lidar measurement error is due to atmospheric variability in the remote-sensing scan volume. This study describes and quantifies the distribution of measurement error due to turbulence in varying atmospheric stability. While the lidar error model is general, we demonstrate the approach using large ensembles of virtual WindCube V2 lidar performing a profiling Doppler-beam-swinging scan in quasi-stationary large-eddy simulations (LESs) of convective and stable boundary layers. Error trends vary with the stability regime, time averaging of results, and observation height. A systematic analysis of the observation error explains dominant mechanisms and supports the findings of the empirical results. Treating the error under a random variable framework allows for informed predictions about the effect of different configurations or conditions on lidar performance. Convective conditions are most prone to large errors (up to 1.5 m s−1 in 1 Hz wind speed in strong convection), driven by the large vertical velocity variances in convective conditions and the high elevation angle of the scanning beams (62∘). Range-gate weighting induces a negative bias into the horizontal wind speeds near the surface shear layer (−0.2 m s−1 in the stable test case). Errors in the horizontal wind speed and direction computed from the wind components are sensitive to the background wind speed but have negligible dependence on the relative orientation of the instrument. Especially during low winds and in the presence of large errors in the horizontal velocity estimates, the reported wind speed is subject to a systematic positive bias (up to 0.4 m s−1 in 1 Hz measurements in strong convection). Vector time-averaged measurements can improve the behavior of the error distributions (reducing the 10 min wind speed error standard deviation to <0.3 m s−1 and the bias to <0.1 m s−1 in strong convection) with a predictable effectiveness related to the number of decorrelated samples in the time window. Hybrid schemes weighting the 10 min scalar- and vector-averaged lidar measurements are shown to be effective at reducing the wind speed biases compared to cup measurements in most of the simulated conditions, with time averages longer than 10 min recommended for best use in some unstable conditions. The approach in decomposing the error mechanisms with the help of the LES flow field could be extended to more complex measurement scenarios and scans.
Analysis of floating-point round-off error in linear algebra routines for graph clustering
Explore the source record for details and available documents.
Novel Radiobiological Gamma Index for Evaluation of 3-Dimensional Predicted Dose Distribution
Purpose: To propose a gamma index-based dose evaluation index that integrates the radiobiological parameters of tumor control (TCP) and normal tissue complication probabilities (NTCP). Methods and Materials: Fifteen prostate and head and neck (H&N) cancer patients received intensity modulated radiation therapy. Before treatment, patient-specific quality assurance was conducted via beam-by-beam analysis, and beam-specific dose error distributions were generated. The predicted 3-dimensional (3D) dose distribution was calculated by back-projection of relative dose error distribution per beam. A 3D gamma analysis of different organs (prostate: clinical [CTV] and planned target volumes [PTV], rectum, bladder, femoral heads; H&N: gross tumor volume [GTV], CTV, spinal cord, brain stem, both parotids) was performed using predicted and planned dose distributions under 2%/2 mm tolerance and physical gamma passing rate was calculated. TCP and NTCP values were calculated for voxels with physical gamma indices (PGI) >1. We propose a new radiobiological gamma index (RGI) to quantify the radiobiological effects of TCP and NTCP and calculate radiobiological gamma passing rates. Results: The mean RGI gamma passing rates for prostate cases were significantly different compared with those of PGI (P<.03–.001). The mean RGI gamma passing rates for H&N cases (except for GTV) were significantly different compared with those of PGI (P<.001). Differences in gamma passing rates between PGI and RGI were due to dose differences between the planned and predicted dose distributions. Radiobiological gamma distribution was visualized to identify areas where the dose was radiobiologically important. Conclusions: RGI was proposed to integrate radiobiological effects into PGI. This index would assist physicians and medical physicists not only in physical evaluations of treatment delivery accuracy, but also in clinical evaluations of predicted dose distribution.
Mechanism reduction for multicomponent surrogates: A case study using toluene reference fuels
Explore the source record for details and available documents.
SU-F-T-308: Mobius FX Evaluation and Comparison Against a Commercial 4D Detector Array for VMAT Plan QA
Purpose: To evaluate the use of MobiusFX as a pre-treatment verification IMRT QA tool and compare it with a commercial 4D detector array for VMAT plan QA. Methods: 15 VMAT plan QA of different treatment sites were delivered and measured by traditional means with the 4D detector array ArcCheck (Sun Nuclear corporation) and at the same time measurement in linac treatment logs (Varian Dynalogs files) were analyzed from the same delivery with MobiusFX software (Mobius Medical Systems). VMAT plan QAs created in Eclipse treatment planning system (Varian) in a TrueBeam linac machine (Varian) were delivered and analyzed with the gamma analysis routine from SNPA software (Sun Nuclear corporation). Results: Comparable results in terms of the gamma analysis with 99.06% average gamma passing with 3%,3mm passing rate is observed in the comparison among MobiusFX, ArcCheck measurements, and the Treatment Planning System dose calculated. When going to a stricter criterion (1%,1mm) larger discrepancies are observed in different regions of the measurements with an average gamma of 66.24% between MobiusFX and ArcCheck. Conclusion: This work indicates the potential for using MobiusFX as a routine pre-treatment patient specific IMRT method for quality assurance purposes and its advantages as a phantom-less method which reduce the time for IMRT QA measurement. MobiusFX is capable of produce similar results of those by traditional methods used for patient specific pre-treatment verification VMAT QA. Even the gamma results comparing to the TPS are similar the analysis of both methods show that the errors being identified by each method are found in different regions. Traditional methods like ArcCheck are sensitive to setup errors and dose difference errors coming from the linac output. On the other hand linac log files analysis record different errors in the VMAT QA associated with the MLCs and gantry motion that by traditional methods cannot be detected.
Minimizing treatment planning errors in proton therapy using failure mode and effects analysis
Purpose: Failure mode and effects analysis (FMEA) is a widely used tool to evaluate safety or reliability in conventional photon radiation therapy. However, reports about FMEA application in proton therapy are scarce. The purpose of this study is to apply FMEA in safety improvement of proton treatment planning at their center. Methods: The authors performed an FMEA analysis of their proton therapy treatment planning process using uniform scanning proton beams. The authors identified possible failure modes in various planning processes, including image fusion, contouring, beam arrangement, dose calculation, plan export, documents, billing, and so on. For each error, the authors estimated the frequency of occurrence, the likelihood of being undetected, and the severity of the error if it went undetected and calculated the risk priority number (RPN). The FMEA results were used to design their quality management program. In addition, the authors created a database to track the identified dosimetric errors. Periodically, the authors reevaluated the risk of errors by reviewing the internal error database and improved their quality assurance program as needed. Results: In total, the authors identified over 36 possible treatment planning related failure modes and estimated the associated occurrence, detectability, and severity to calculate the overall risk priority number. Based on the FMEA, the authors implemented various safety improvement procedures into their practice, such as education, peer review, and automatic check tools. The ongoing error tracking database provided realistic data on the frequency of occurrence with which to reevaluate the RPNs for various failure modes. Conclusions: The FMEA technique provides a systematic method for identifying and evaluating potential errors in proton treatment planning before they result in an error in patient dose delivery. The application of FMEA framework and the implementation of an ongoing error tracking system at their clinic have proven to be useful in error reduction in proton treatment planning, thus improving the effectiveness and safety of proton therapy.
Method for analyzing passive silicon carbide thermometry with a continuous dilatometer to determine irradiation temperature
Silicon carbide is used as a passive post-irradiation temperature monitor because the irradiation defects will anneal out above the irradiation temperature. The irradiation temperature is determined by measuring a property change after isochronal annealing, i.e., lattice spacing, dimensions, electrical resistivity, thermal diffusivity, or bulk density. However, such methods are time-consuming since the steps involved must be performed in a serial manner. This work presents the use of thermal expansion from continuous dilatometry to calculate the SiC irradiation temperature, which is an automated process requiring minimal setup time. Analysis software was written that performs the calculations to obtain the irradiation temperature and removes possible user-introduced error while standardizing the analysis. In addition, this method has been compared to an electrical resistivity and isochronal annealing investigation, and the results revealed agreement of the calculated temperatures. These results show that dilatometry is a reliable and less time-intensive process for determining irradiation temperature from passive SiC thermometry.
Accurate description of torsion potentials in conjugated polymers using density functionals with reduced self-interaction error
We investigate the torsion potentials in two prototypical π-conjugated polymers, polyacetylene and polydiacetylene, as a function of chain length using different flavors of density functional theory. Our study provides a quantitative analysis of the delocalization error in standard semilocal and hybrid density functionals and demonstrates how it can influence structural and thermodynamic properties. The delocalization error is quantified by evaluating the many-electron self-interaction error (MESIE) for fractional electron numbers, which allows us to establish a direct connection between the MESIE and the error in the torsion barriers. The use of non-empirically tuned long-range corrected hybrid functionals results in a very significant reduction of the MESIE and leads to an improved description of torsion barrier heights. In addition, we demonstrate how our analysis allows the determination of the effective conjugation length in polyacetylene and polydiacetylene chains.
Z-checker: A framework for assessing lossy compression of scientific data
Because of the vast volume of data being produced by today’s scientific simulations and experiments, lossy data compressor allowing user-controlled loss of accuracy during the compression is a relevant solution for significantly reducing the data size. However, lossy compressor developers and users are missing a tool to explore the features of scientific data sets and understand the data alteration after compression in a systematic and reliable way. To address this gap, we have designed and implemented a generic framework called Z-checker. On the one hand, Z-checker combines a battery of data analysis components for data compression. On the other hand, Z-checker is implemented as an open-source community tool to which users and developers can contribute and add new analysis components based on their additional analysis demands. In this article, we present a survey of existing lossy compressors. Then, we describe the design framework of Z-checker, in which we integrated evaluation metrics proposed in prior work as well as other analysis tools. Specifically, for lossy compressor developers, Z-checker can be used to characterize critical properties (such as entropy, distribution, power spectrum, principal component analysis, and autocorrelation) of any data set to improve compression strategies. For lossy compression users, Z-checker can detect the compression quality (compression ratio and bit rate) and provide various global distortion analysis comparing the original data with the decompressed data (peak signal-to-noise ratio, normalized mean squared error, rate–distortion, rate-compression error, spectral, distribution, and derivatives) and statistical analysis of the compression error (maximum, minimum, and average error; autocorrelation; and distribution of errors). Z-checker can perform the analysis with either coarse granularity (throughout the whole data set) or fine granularity (by user-defined blocks), such that the users and developers can select the best fit, adaptive compressors for different parts of the data set. Z-checker features a visualization interface displaying all analysis results in addition to some basic views of the data sets such as time series. To the best of our knowledge, Z-checker is the first tool designed to assess lossy compression comprehensively for scientific data sets.
Robust measurement of microbial reduction of graphene oxide nanoparticles using image analysis
ABSTRACT Shewanella oneidensis ( S. oneidensis ) has the capacity to reduce electron acceptors within a medium and is thus used frequently in microbial fuel generation, pollutant breakdown, and nanoparticle fabrication. Microbial fuel setups, however, often require costly or labor-intensive components, thus making optimization of their performance onerous. For rapid optimization of setup conditions, a model reduction assay can be employed to allow simultaneous, large-scale experiments at lower cost and effort. Since S. oneidensis uses different extracellular electron transfer pathways depending on the electron acceptor, it is essential to use a reduction assay that mirrors the pathways employed in the microbial fuel system. For microbial fuel setups that use nanoparticles to stimulate electron transfer, reduction of graphene oxide provides a more accurate model than other commonly used assays as it is a bulk material that forms flocculates in solutions with a large ionic component. However, graphene oxide flocculates can interfere with traditional absorbance-based measurement techniques. This study introduces a novel image analysis method for quantifying graphene oxide reduction, showing improved performance and statistical accuracy over traditional methods. A comparative analysis shows that the image analysis method produces smaller errors between replicates and reveals more statistically significant differences between samples than traditional plate reader measurements under conditions causing graphene oxide flocculation. Image analysis can also detect reduction activity at earlier time points due to its use of larger solution volumes, enhancing color detection. These improvements in accuracy make image analysis a promising method for optimizing microbial fuel cells that use nanoparticles or bulk substrates. IMPORTANCE Shewanella oneidensis ( S. oneidensis ) is widely used in reduction processes such as microbial fuel generation due to its capacity to reduce electron acceptors. Often, these setups are labor-intensive to operate and require days to produce results, so use of a model assay would reduce the time and expenses needed for optimization. Our research developed a novel digital analysis method for analysis of graphene oxide flocculates that may be utilized as a model assay for reduction platforms featuring nanoparticles. Use of this model reduction assay will enable rapid optimization and drive improvements in the microbial fuel generation sector.
Locality and Digital Quantum Simulation of Power-Law Interactions
The propagation of information in nonrelativistic quantum systems obeys a speed limit known as a Lieb-Robinson bound. We derive a new Lieb-Robinson bound for systems with interactions that decay with distance r as a power law, 1 / r α . The bound implies an effective light cone tighter than all previous bounds. Our approach is based on a technique for approximating the time evolution of a system, which was first introduced as part of a quantum simulation algorithm by Haah et al ., FOCS’18. To bound the error of the approximation, we use a known Lieb-Robinson bound that is weaker than the bound we establish. This result brings the analysis full circle, suggesting a deep connection between Lieb-Robinson bounds and digital quantum simulation. In addition to the new Lieb-Robinson bound, our analysis also gives an error bound for the Haah et al. quantum simulation algorithm when used to simulate power-law decaying interactions. In particular, we show that the gate count of the algorithm scales with the system size better than existing algorithms when α > 3 D (where D is the number of dimensions).
Constraints on Neutrino Oscillation Parameters from Neutrinos and Antineutrinos with Machine Learning
NOvA is a two detector, long baseline neutrino oscillation experiment measuring the oscillations of muon neutrinos from the \numi neutrino beam over a baseline of \SI{810}{km}. The experiment uses four oscillation channels, $\numu \rightarrow \numu$, $\numubar \rightarrow \numubar$, $\numu \rightarrow \nue$, and $\numubar \rightarrow \nuebar$, with a peak neutrino energy of \SI{1.8}{GeV}. This dissertation describes the analysis of these channels using a dataset of $13.6\times10^{20}$ protons on target neutrino beam mode and $12.5\times10^{20}$ protons on target antineutrino beam mode. The analysis makes use of improvements in the treatment of systematic uncertainties and machine learning techniques to reconstruct neutrino interactions. A technique for decorrelating systematic errors using principle component analysis was utilized to reduce and optimize neutrino cross section and beam related uncertainties. The improved machine learning algorithms make use of convolutional ne ural net works for neutrino event classification, particle classification, and instance segmentation. The selection of neutrino signal events utilizing the neutrino event classifier shows an efficiency of 63\% for the selection of electron neutrinos in neutrino beam mode and 75\% for electron antineutrinos in antineutrino beam mode. Using this algorithm, 82 appearing electron neutrino candidates and 33 appearing electron antineutrino candidates were observed with expected backgrounds of 26.8 and 14.0 respectively. In addition, 211 surviving muon neutrino candidates and 105 muon antineutrino candidates were identified with a purity of more than 96\% using the same neutrino event classifier. Fitting these data to the three flavor neutrino oscillation model, using constraints on \thetaonetwo, \thetaonethree, and \dmsqonetwo from solar and reactor neutrino experiments, the oscillation parameters are measured to be $\sintwothree = 0.57^{+0.04}_{-0.03}$, $\dmsqthreetwo = \SI[parse-numbers= false]{+ 2.41\pm0.07 \times 10^{-3}}{eV^2}$, and $\dcp=0.82^{+0.27}_{-0.87}\pi$ with a preference for the normal neutrino mass hierarchy. Leading systematic uncertainties for these measurements come from detector calibration and neutrino interaction models.
Addendum: Unified framework for open quantum dynamics with memory
This Addendum presents a detailed analysis of the discretization error in time-integration and time-derivative that appear in the Nakajima-Zwanzig equation. This was brought to our attention by Makri et al. [arXiv:2410.08239]. Our analysis in the Addendum shows that the relationship derived in our earlier work [Nat. Commun. 15, 8087 (2024)] is valid within the choice of discretization and is not contaminated by the discretization error.
Finite-size effects in periodic coupled cluster calculations
Here, we provide the first rigorous study of the finite-size error in the simplest and representative coupled cluster theory, namely the coupled cluster doubles (CCD) theory, for gapped periodic systems. Given exact Hartree-Fock orbitals and their corresponding orbital energies, we demonstrate that the correlation energy obtained from the approximate CCD method, after a finite number of fixed-point iterations over the amplitude equation, exhibits a finite-size error scaling as $\mathcal{O}(N^{-\frac{1}{3}}_k)$. Here $N_k$ is the number of discretization points in the Brillouin zone and characterizes the system size. Under additional assumptions ensuring the convergence of the fixed-point iterations, we demonstrate that the CCD correlation energy also exhibits a finite-size error scaling as $\mathcal{O}(N^{-\frac{1}{3}}_k)$. Our analysis shows that the dominant error lies in the coupled cluster amplitude calculation, and the convergence of the finite-size error in energy calculation can be boosted to $\mathcal{O}(N^{-1}_k)$ with accurate amplitudes. This also provides the first proof of the scaling of the finite-size error in the third order Møller-Plesset perturbation theory (MP3) for periodic systems.
Three-beam rotational coherent anti-Stokes Raman spectroscopy thermometry in scattering environments
Three-beam rotational coherent anti-Stokes Raman scattering (CARS) measurements performed in highly scattering environments are susceptible to contamination by two-beam CARS signals generated by the pump–probe and Stokes–probe interactions at the measurement volume. If this occurs, differences in the Raman excitation bandwidth between the two-beam and three-beam CARS signals can add significant errors to the spectral analysis. This interference to the best of our knowledge has not been acknowledged in previous three-beam rotational CARS experiments, but may introduce measurement errors up to 25% depending on the temperature, amount of scattering, and differences between the two-beam and three-beam Raman excitation bandwidths. Here, the presence of two-beam CARS signal contamination was experimentally verified using a femtosecond–picosecond rotational CARS instrument in two scattering environments: (1) a fireball generated by a laboratory-scale explosion that contained particulate matter, metal fragments, and soot, and (2) a flow of air and small liquid droplets. A polarization scheme is presented to overcome this interference. By rotating the pump and Stokes polarizations +55° and –55° from the probe, respectively, the two-beam and three-beam CARS signals are orthogonally polarized and can be separated using a polarization analyzer. Using this polarization arrangement, the Raman-resonant three-beam CARS signal amplitude is reduced by a factor of 2.3 compared to the case where all polarizations are parallel. This method is successfully demonstrated in both scattering environments. A theoretical model is presented, and the temperature measurement error is studied for different experimental conditions. The criteria for when this interference may be present are discussed.
Numerical study of the effect of normalised window size, sampling frequency, and noise level on short time Fourier transform analysis
Photonic Doppler velocimetry, also known as heterodyne velocimetry, is a widely used optical technique that requires the analysis of frequency modulated signals. This paper describes an investigation into the errors of short time Fourier transform analysis. The number of variables requiring investigation was reduced by means of an equivalence principle. Error predictions, as the number of cycles, samples per cycle, noise level, and window type were varied, are presented. The results were found to be in good agreement with analytical models.