Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Confidence”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Assessing decadal variability of subseasonal forecasts of opportunity using explainable AI

Abstract Identifying predictable states of the climate system allows for enhanced prediction skill on the generally low-skill subseasonal timescale via forecasts with higher confidence and accuracy, known as forecasts of opportunity. This study takes a neural network approach to explore decadal variability of subseasonal predictability, particularly during forecasts of opportunity. Specifically, this work quantifies subseasonal prediction skill provided by the tropics within the Community Earth System Model Version 2 (CESM2) Large Ensemble and assesses how this skill evolves on decadal timescales. Utilizing the networks’ confidence and explainable artificial intelligence, physically meaningful sources of predictability associated with periods of enhanced skill are identified. Using these networks, we find that tropically-driven subseasonal predictability varies on decadal timescales during forecasts of opportunity. Further, we investigate the drivers of the low frequency modulation of the tropical-extratropical teleconnection and discuss the implications. Analysis is extended to ECMWF Reanalysis v5 data, revealing that the relationships learned within the CESM2-Large Ensemble holds in modern reanalysis data. These results indicate that the neural networks are capable of identifying predictable decadal states of the climate system within CESM2 that are useful for making confident, accurate subseasonal precipitation predictions in the real world.

54 ENVIRONMENTAL SCIENCES↗

VPF-Class: taxonomic assignment and host prediction of uncultivated viruses based on viral protein families

Abstract Motivation Two key steps in the analysis of uncultured viruses recovered from metagenomes are the taxonomic classification of the viral sequences and the identification of putative host(s). Both steps rely mainly on the assignment of viral proteins to orthologs in cultivated viruses. Viral Protein Families (VPFs) can be used for the robust identification of new viral sequences in large metagenomics datasets. Despite the importance of VPF information for viral discovery, VPFs have not yet been explored for determining viral taxonomy and host targets. Results In this work, we classified the set of VPFs from the IMG/VR database and developed VPF-Class. VPF-Class is a tool that automates the taxonomic classification and host prediction of viral contigs based on the assignment of their proteins to a set of classified VPFs. Applying VPF-Class on 731K uncultivated virus contigs from the IMG/VR database, we were able to classify 363K contigs at the genus level and predict the host of over 461K contigs. In the RefSeq database, VPF-class reported an accuracy of nearly 100% to classify dsDNA, ssDNA and retroviruses, at the genus level, considering a membership ratio and a confidence score of 0.2. The accuracy in host prediction was 86.4%, also at the genus level, considering a membership ratio of 0.3 and a confidence score of 0.5. And, in the prophages dataset, the accuracy in host prediction was 86% considering a membership ratio of 0.6 and a confidence score of 0.8. Moreover, from the Global Ocean Virome dataset, over 817K viral contigs out of 1 million were classified. Availability and implementation The implementation of VPF-Class can be downloaded from https://github.com/biocom-uib/vpf-tools. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗

Experimental Characterization Test of a Grid-Forming Inverter for Microgrid Applications

Standardized experimental testing protocols for grid forming (GFM) inverters to ensure expected operation under both normal and contingency conditions do not exist. Such protocols increase the confidence of system owner/operators that an inverter deployed in a proposed system will engage in typical behaviors to ensure interoperability with other units and ancillary equipment (e.g. protection equipment). This paper presents systematic and comprehensive test protocols to evaluate the performance of GFM inverters under the following operational configurations: islanded operation, heterogeneous islanded operation (parallel with a synchronous generator), grid-connected operation, and transition operation. A commercial GFM inverter is used to verify the test protocols and to understand the inverter's performance and functionalities. In particular, required configuration and tuning of the inverter will be explained in the full paper to enrich the testing protocol.

black start↗

Improved Gas Plume Identification Using Nearest Neighbor Methods for Background Estimation

Longwave infrared (LWIR) hyperspectral imaging (HSI) can be used for many tasks in remote sensing, including detecting and identifying effluent gases by LWIR sensors on airborne platforms. Identification is used after detection to increase confidence in weakly detected plumes, reduce false positives from detection, and distinguish between similar and confounding material signatures. Background estimation is an important step used to reveal the unique spectral characteristics of the detected gas, allowing the identification model to determine what the gas is specifically. The importance of proper background estimation increases when dealing with weak signals, large libraries of gases of interest, and uncommon or heterogeneous backgrounds. In this article, we propose two methods for background estimation: a novel k-nearest segments (KNS) algorithm and the standard k-nearest neighbors (KNN) algorithm. We test our methods and three existing background estimation methods for comparison against global background estimation to determine which performs best at estimating the true background radiance under a plume and for increasing identification confidence using a neural network classification model. We compare the different methods using 640 simulated weak plumes in an urban environment. For identification, our KNS algorithm improves median neural network identification confidence by 53.2%. For background radiance estimation, the KNN algorithm provides a median of 49 times less RMSE than global background estimation. Furthermore, KNN is the easiest method to tune for different plumes, making it an excellent “out of the box” background estimator.

47 OTHER INSTRUMENTATION↗

Direct constraint on the Higgs–charm coupling from a search for Higgs boson decays into charm quarks with the ATLAS detector

A search for the Higgs boson decaying into a pair of charm quarks is presented. The analysis uses proton–proton collisions to target the production of a Higgs boson in association with a leptonically decaying W or Z boson. The dataset delivered by the LHC at a centre-of-mass energy of $\sqrt{s}$ = 13 TeV and recorded by the ATLAS detector corresponds to an integrated luminosity of 139 fb -1 . Flavour-tagging algorithms are used to identify jets originating from the hadronisation of charm quarks. The analysis method is validated with the simultaneous measurement of WW, WZ and ZZ production, with observed (expected) significances of 2.6 (2.2) standard deviations above the background-only prediction for the $(W/Z)Z( → c\bar{c})$ process and 3.8 (4.6) standard deviations for the $(W/Z)W(→cq)$ process. The $(W/Z)Z( → c\bar{c})$ search yields an observed (expected) upper limit of 26 (31) times the predicted Standard Model cross-section times branching fraction for a Higgs boson with a mass of 125 GeV, corresponding to an observed (expected) constraint on the charm Yukawa coupling modifier |$k_c$| < 8.5 (12.4), at the 95% confidence level. A combination with the ATLAS $(W/Z)H,H → b\bar{b}$ analysis is performed, allowing the ratio $k_c/k_b$ to be constrained to less than 4.5 at the 95% confidence level, smaller than the ratio of the b- and c-quark masses, and therefore determines the Higgs-charm coupling to be weaker than the Higgs-bottom coupling at the 95% confidence level.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Resilience Measurement Framework For Post-deployment Artificial Intelligence (ai) Integrated Systems

Resilience is largely defined as the ability to adapt or recover from adverse conditions, stresses, attacks, or compromises on systems that use or are enabled by digital resources. In Artificial Intelligence Management and Research for Advanced Networked Testbed Hub (AMARANTH), resilience is measured in the amount of time it took from the beginning of a testing period for the model to reach predictions outside of the original 95% confidence interval or using the Kullback-Leibler (KL) divergence theorem, the Population Stability Index (PSI), and traditional methods such as root mean squared error (RMSE) threshold. Artificial Intelligence (AI) model drift is of significant concern when deploying AI-integrated systems into critical and/or secure environments. Drift can impact resilience of the AI-integrated system post-deployment and requires consistent maintenance and upkeep to ensure the model is accurate and precise. To quantify model drift and predict the point when a model's drift becomes unacceptable, we describe using Kullback-Leibler (KL) divergence, Population Stability Index (PSI) and/or confidence interval width estimations to determine the point of failure and time to failure of a model post-deployment. Through simple code functions, the KL-divergence, PSI, confidence interval, and root mean squared (RMSE) point of failures can be used to derive when a model needs to be maintained as well as the impact of adversarial action through statistical means.

Yockey, Patience [Idaho National Laboratory (INL),↗

Evaluation of an autonomous acoustic surveying technique for grassland bird communities in Nebraska

Monitoring trends in wildlife communities is integral to making informed land management decisions and applying conservation strategies. Birds inhabit most niches in every environment and because of this they are widely accepted as an indicator species for environmental health. Traditionally, point counts are the common method to survey bird populations, however, passive acoustic monitoring approaches using autonomous recording units have been shown to be cost-effective alternatives to point count surveys. Advancements in automatic acoustic classification technologies, such as BirdNET, can aid in these efforts by quickly processing large volumes of acoustic recordings to identify bird species. While the utility of BirdNET has been demonstrated in several applications, there is little understanding of its effectiveness in surveying declining grassland birds. We conducted a study to evaluate the performance of BirdNET to survey grassland bird communities in Nebraska by comparing this automated approach to point count surveys. We deployed ten autonomous recording units from March through September 2022: five recorders in row-crop fields and five recorders in perennial grassland fields. During this study period, we visited each site three times to conduct point count surveys. We compared focal grassland bird species richness between point count surveys and the autonomous recording units at two different temporal scales and at six different confidence thresholds. Total species richness (focal and non-focal) for both methods was also compared at five different confidence thresholds using species accumulation curves. The results from this study demonstrate the usefulness of BirdNET at estimating long-term grassland bird species richness at default confidence scores, however, obtaining accurate abundance estimates for uncommon bird species may require validation with traditional methods.

59 BASIC BIOLOGICAL SCIENCES↗

Defense Waste Processing Facility Nitric-Glycolic Flowsheet Chemical Process Cell Chemistry: Part 2

The conversions of nitrite to nitrate, the destruction of glycolate, and the conversion of glycolate to formate and oxalate were modeled for the Nitric-Glycolic flowsheet using data from Chemical Process Cell (CPC) simulant runs conducted by Savannah River National Laboratory (SRNL) from 2011 to 2016. The goal of this work was to develop empirical correlation models to predict these values from measurable variables from the chemical process so that these quantities could be predicted a-priori from the sludge or simulant composition and measurable processing variables. The need for these predictions arises from the need to predict the REDuction/OXidation (REDOX) state of the glass from the Defense Waste Processing Facility (DWPF) melter. This report summarizes the work on these correlations based on the aforementioned data. Previous work on these correlations was documented in a technical report covering data from 2011-2015. This current report supersedes this previous report. Further refinement of the models as additional data are collected is recommended. The glass REDOX depends on the concentrations of nitrate and manganese (oxidants), and of glycolate, formate, oxalate, carbon, and antifoam (reductants) in the melter feed. The waste sludge contains nitrite, nitrate, manganese (Mn), and oxalate. Virtually all of the nitrite is converted to nitrate or NO+NO 2 +N 2 O gases in the CPC. The portion of the nitrite converted to nitrate increases the amount of nitrate in the sludge. The amount of glycolate in the final melter feed depends on the amount of the glycolic acid feed that is destroyed. Similarly, the amounts of formate and oxalate formed during the decomposition of glycolic acid are required. The material balance on carbon was found to not close in most cases. Generally, there was less carbon at the end of testing compared to the inputs. The most uncertain product variable was glycolate, so material balances were performed where the glycolate concentration was adjusted, usually upward, to close the balance. Correlation versus the original, as-measured, data was generally poor, but correlation against the material balance adjusted values was greatly improved. It was also shown that the correlation of the measured REDOX versus the predicted REDOX was much better when the material balance adjusted glycolate values were used. Three data series were primarily used during the regressions of the data; these series were 1) Sludge Batch 9 NG flowsheet simulant runs NG51-62 (SB9-NG); 2) Scaled Runs + Bounding Hydrogen Runs (SR+BH); and 3) Runs GN43-50 and 57 (43-50,57). The glycolate destruction was found to correlate with acid stoichiometry (AS), percent reducing acid (PRA), and for some data series, headspace to simulant volume ratio (HSV), mercury (Hg), and nitrate. Although glycolate destruction for pairs of data series (e.g., [SB9-NG] and [SR+BH]) were found to depend on HSV, the combination of all three data series was not found to have significant dependence on this variable. The best model for glycolate destruction depended on AS, nitrate, and Hg. This model predicted the product glycolate compositions of the data to within 92-106%. The conversion of glycolate to formate was high when noble metals and Hg were not present, with values up to 100%. When noble metals and Hg were present, this conversion ranged from zero to 7%, and was dependent on AS. Lower AS gave higher conversions to formate. The conversion to oxalate was found to depend on the AS and the initial concentration of nitrite. An alternative fit versus AS and the form of ruthenium (Ru) used is a possible alternative. This fit was somewhat less statistically significant. This second model predicts that more oxalate is formed when Ru-nitrosyl nitrate is used rather than Ru chloride. The conversion of glycolate to oxalate ranged from zero to 6%. The conversion of nitrite to nitrate depended primarily on AS and PRA, with HSV and Hg being significant when these variables were varied. For multiple series of data, nitrite was also needed to SRNL-STI-2017-00172 5HYLVLRQ viL distinguish between data series, and the effect of HSV became insignificant. The best model for nitrite to nitrate conversion depended on AS, PRA, nitrite, and Hg. The 95% confidence intervals on the predicted values of glycolate destruction, glycolate to oxalate conversion, and nitrite to nitrate conversion were used to determine the uncertainty in the predicted REDOX when starting with only the composition of the sludge, AS, and PRA. Using the 95% confidences on an individual value (that is the confidence in getting a particular value for one single test as opposed to what the mean would be for multiple tests), the uncertainty in the predicted REDOX was calculated. The uncertainty in the actual product composition glycolate, oxalate, formate, and nitrate concentrations translated to an uncertainty in the REDOX value of ±0.1,which is approximately the uncertainty claimed in the REDOX model itself.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Quantifying Uncertainty in All-to-All Estimates of Space Object Conjunction Probabilities using U-Statistics

Predicting space object conjunctions is inherently probabilistic due to initial state and orbit model uncertainty. A commonly considered Monte Carlo estimator of the conjunction probability is the ’all-to-all’ estimator. Given independent random samples of the trajectories of both objects, the estimator is the percentage of all pairs of trajectories that result in a conjunction. Intuitively, the all-to-all estimator is the best possible estimator of the conjunction probability since it considers all pairs of Monte Carlo samples. However, its distribution is not available in closed-form, which limits its use in practice and makes this intuition difficult to make rigorous. In this paper, the all-to-all estimator is identified as a U-statistic, which implies that it has several favorable properties. Specifically, the estimator is the minimum variance unbiased estimator of the conjunction probability and is asymptotically Gaussian distributed. An approximate confidence interval for the conjunction probability is obtained from an estimate of the asymptotic Gaussian distribution. We show how to efficiently compute the confidence interval and demonstrate that the interval has the nominal coverage level. The confidence intervals are also seen to be narrower than those based on the commonly-used each-to-each estimator. Furthermore, the all-to-all estimator is shown to allow different Monte Carlo sample sizes, whereas the each-to-each estimator requires equal sample sizes.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

TRUST Contact Thermal Conductance (CTC) Report

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainties in Simulations and Tests (TRUST) work package is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are critical to the current and future stockpile. Staff development will include cross-discipline training to provide engineers with experience in both numerical simulations and experimental methods. This work will use and provide feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development: engineering common model framework (ECMF), engineering quantification of margins and uncertainties (EQMU), and the test information management system (TIMS).

42 ENGINEERING↗

Sensitivity and Uncertainty of the IFR-1 BISON Benchmark

The fuel performance code BISON is being used to evaluate metallic fuel for a new fast-spectrum test reactor called the Versatile Test Reactor, which is being considered by the US Department of Energy. To quantify the accuracy of BISON predictions, researchers at Oak Ridge National Laboratory have been developing a series of benchmarks based on legacy metallic fuel experiments. As part of this effort, the sensitivity of BISON predictions to variations in model inputs and the uncertainties associated with BISON predictions must be established. This report summarizes efforts to perform a comprehensive sensitivity analysis (SA) and uncertainty quantification (UQ) on a benchmark based on the IFR-1 experiment. For the SA, at least one input was chosen from every BISON model and physics module used in the benchmark. The inputs were varied individually in a series of BISON simulations. The resulting variations in benchmark predictions were normalized to calculate sensitivities. The strongest sensitivities were identified and used to inform input selections for the UQ. The UQ was performed using the Monte Carlo UQ method. A literature review was conducted to estimate uncertainty distributions for the selected inputs, and values were sampled randomly from each distribution in a series of BISON simulations. Variations in the benchmark predictions were used to estimate uncertainty distributions and confidence intervals. It was found that nearly 100% of benchmark predictions matched the corresponding legacy values within the confidence intervals. However, this is at least partially because of the wide confidence intervals associated with the benchmark predictions. The uncertainty contributions of assumptions in the benchmark, experimental uncertainties, and BISON models were quantified. Some analysis was performed to identify inputs that contributed to the uncertainties. Finally, recommendations are made for future benchmark development and future BISON development.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

TRUST End-of-Year Report

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainties in Simulations and Tests (TRUST) work package is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are critical to the current and future stockpile. Staff development will include cross-discipline training to provide engineers with experience in both numerical simulations and experimental methods. This work will use and provide feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development: engineering common model framework (ECMF), engineering quantification of margins and uncertainties (EQMU), and the test information management system (TIMS). TRUST includes four testbeds and their associated engineering analysis baseline models (EABMs): 1. contact thermal conductivity (CTC); 2. nonlinear dynamics (ND); 3. sensors in environments for accelerometers (SEA); 4. sensors in environments for fiber optic displacement gages (SEFOD).

42 ENGINEERING↗

TRUST Nonlinear Dynamics (ND) Report

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainties in Simulations and Tests (TRUST) work package is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are critical to the current and future stockpile. Staff development will include cross-discipline training to provide engineers with experience in both numerical simulations and experimental methods. This work will use and provide feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development: engineering common model framework (ECMF), engineering quantification of margins and uncertainties (EQMU), and the test information management system (TIMS).

42 ENGINEERING↗

Oppenheimer Science and Energy Leadership Program (OSELP) Simulation and Computation at LANL [Slides]

Simulation and Computation plays a pivotal role in maintaining confidence in the stockpile as the approach to underwrite that confidence has changed. The need for resolution and fidelity at scale drives our need for increased computing capability. DOE/NNSA Advanced Simulation and Computing (ASC) program provides the computational surrogate for testing. ASC provides simulation-based confidence in the U.S. stockpile.

97 MATHEMATICS AND COMPUTING↗

TRUST Contact Thermal Conductance (TRUST-CTC) Report: FY2022

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainties in Simulations and Tests (TRUST) work package is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are critical to the current and future stockpile. Staff development will include cross-discipline training to provide engineers with experience in both numerical simulations and experimental methods. This work will use and provide feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development: engineering common model framework (ECMF), engineering quantification of margins and uncertainties (EQMU), test information management system (TIMS), and [WAVES] Analysis for Verified Engineering Simulation (WAVES).

42 ENGINEERING↗

Development of a Gibbs Energy Minimiser for the MOOSE-based Corrosion Modelling App Yellowjacket and Validation of MSTDB

Nuclear materials are highly complex multiscale, multiphysics systems,and an effective prediction of nuclear reactor performance and safety requires simulation capabilities that tightly couple different physical phenomena. The Idaho National Laboratory’s Multiphysics Object Oriented Simulation Environment (MOOSE) provides the computational foundation for performing such simulations. With the move towards advanced reactors, such as the Molten Salt Reactor (MSR), that employ high temperature fluids compared to conventional reactors, corrosion has become a problem of great interest. A new application called Yellowjacket is currently under development to directly couple thermodynamic equilibrium and kinetics with phase field models in order to model corrosion in MSRs. As part of Yellowjacket, a Gibbs energy minimiser is being developed to perform thermochemical equilibrium calculations for a range of different materials, which is currently in its infancy. This report describes the further progress towards the development of Yellowjacket Gibbs energy minimiser. Ontario Tech University is developing a new Gibbs energy minimiser for Yellowjacket which is the primary contribution of this work. The aim to develop a thermochemistry solver for the MOOSE framework following the same development philosophy and using the same tools and libraries. A special focus is on performance, documentation and SQA. Furthermore through a scope extension partway through the fiscal year a thorough assessment of the MSTDB-TC v1.3 was performed at Ontario Tech University in the context of continuous improvement and quality assurance. The objective of the work was to have an arm’s length review of the database to give confidence that the database is performing as it was intended while assessing its current state to give recommendations to future developments. This assessment involved two parts: A) a quantitative assessment, and B) a qualitative assessment. Part A involved developing an automated test-suite that would compute values from the database using Thermochimica with comparisons to experimental measurements for validation purposes, which gives confidence to the database’s stakeholders that its functioning properly. Part B involved reviewing all binary systems in the database and making a qualitative assessment with two performance indicators: comprehensiveness and overall confidence. It is important to note that the models in the database are empirical, which is to say that the quality of any model is highly dependent on the experimental data used to inform its development.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

TRUST Nonlinear Dynamics (ND) Report: FY22

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainty in Simulations and Tests (TRUST) work package, is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are relevant to the current and future stockpile. This work will use and provide feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development: engineering common model framework (ECMF), engineering quantification of margins and uncertainties (EQMU), and the test information management system (TIMS). Additionally, this work is being used to test a newly available framework in W-13, Weapons Analysis for Verified Engineering Simulation (WAVES).

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

TRUST Contact Thermal Conductance (TRUST-CTC) Report: FY23

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainties in Simulations and Tests (TRUST) work package is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are critical to the current and future stockpile. Staff development will include cross-discipline training to provide engineers with experience in both numerical simulations and experimental methods. This work will use and provide feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development: engineering common model framework (ECMF), engineering quantification of margins and uncertainties (EQMU), test information management system (TIMS), and [WAVES] Analysis for Verified Engineering Simulation (WAVES).

42 ENGINEERING↗