Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

2014 Groundwater Monitoring Report Central Nevada Test Area Subsurface Corrective Action Unit 443

The Central Nevada Test Area was the site of a 0.2- to 1-megaton underground nuclear test in 1968. The surface of the site has been closed, but the subsurface is still in the corrective action process. The corrective action alternative selected for the site was monitoring with institutional controls. Annual sampling and hydraulic head monitoring of the well network are conducted as part of the subsurface corrective action strategy for the site. The monitoring well network consists of 12 locations (wells and piezometers) in the alluvium, seven locations (wells and piezometers) in the volcanic section, and one well in the collapse chimney that resulted from the underground nuclear test. The site is currently in the conceptual model evaluation phase of the corrective action strategy.

54 ENVIRONMENTAL SCIENCES↗

SMART Deliverable 6.1.2a: Application of the ORION tool to the IBDP Carbon Storage Site

Forecasting and managing potential induced seismic activity is one of the challenges facing commercialscale geologic carbon sequestration (GCS), as well as other geologic energy extraction and byproduct disposal technologies. Historically, the process to develop robust, science-based forecasts of induced seismicity has required an integrated effort from experts in seismology, geomechanics, and reservoir engineering to manage data, develop and evaluate models of subsurface processes, and to calibrate and interpret the results from a range of models to understand site behavior relative to prescribed standards and in the context of uncertainty in geologic characterization data, forecasting models, and operational scenario uncertainty. The Operational Forecasting of Induced Seismicity (ORION) toolkit is an open-source, observation-based forecasting toolkit that is being co-developed by two U.S. DOE-funded initiatives: the National Risk Assessment Partnership (NRAP) and the Scienceinformed Machine Learning for Accelerating Real Time Decisions in Subsurface Applications (SMART) Initiative. ORION is designed to provide functionality to support decision making about seismic hazard analysis and risk management for GCS stakeholders ranging from the public to site operators to expert seismologists. The tool, which is written as open-source code in the Python programming language, is composed of a desktop graphical user interface (GUI) and an underlying forecasting engine. The forecasting engine uses available reservoir properties, well and fluid injection scenario details, and observed seismic catalog data as inputs to produce a set of temporal and spatio-temporal seismic forecasts.

58 GEOSCIENCES↗

Framework for Characterizing Strain in the Vicinity of Leaky Reservoirs

The identification of leaks in the vicinity of reservoirs is critical to determining the feasibility of long-term geologic storage of CO2, methane, wastewater and other fluids. We have established a series of basic reservoir leakage scenarios and performed numerical analysis to characterize the associated pressure and strain fields created from injection into these leaky reservoir settings, as well from ambient loads, such as barometric pressure and earth tides. The end-member conceptual models evaluated include injection and ambient loads applied to: 1) a thin, horizontal layer with no leakage pathways (baseline), 2) a thin layer with a vertical conduit to the surface (e.g., a leaky fault or abandoned well) and 3) a thin layer with a conduit to an overlying aquifer. The pressure and strain fields are sensitive to the reservoir and confining unit geometry, material properties, the distribution, composition and compressibility of the pore fluid and the type of applied load (e.g., constant vs. cyclical injection, barometric pressure fluctuations, earth tides, etc.). Initial results from the numerical analyses indicate that an induced pressure change of 1 MPa in a formation at 1 km depth with a Young’s modulus of 10 GPa is sufficient to create vertical and horizontal strain signals as high as 2 με at a depth of 100 m. The signals predicted from leaky conditions compared to an intact reservoir differ by as much as 0.2 με in magnitude and evolve at different rates. We have developed instruments capable of measuring vertical and horizontal strain at a resolution of better than 5 nε and have recorded repeatable strain signals at 30 m depth in response to wastewater injection into the Avant field in Oklahoma. Based on these findings, strain signals predicted in the shallow subsurface (< 100 m) with numerical analysis are expected to be measurable, and this signal may be inverted to characterize the location and magnitude of leaks, as well as to characterize the geometry and hydromechanical properties of the formation and overlying confining units. This suggests that shallow strain measurements could be a cost effective method for characterizing and monitoring gas storage projects.

Robert, Moak↗

Efficient Learning of Accurate Surrogates for Simulations of Complex Systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse, or dynamic data. We introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation of state model can be reliably auto-generated from expensive calculations using few model evaluations.

97 MATHEMATICS AND COMPUTING↗

Silicon Carbide Composite Cladding Qualification

This report examines the process to qualify a new fuel-cladding type focusing on the data requirements and types of irradiation testing necessary to meet the regulatory burden and related guidance applicable to fuel qualification of accident-tolerant fuel. The report considers the use of accelerated fuel qualification techniques and lead-test specimen programs that may shorten the timeline for qualifying fuel for use in a nuclear reactor at the desired burnup. The assessment framework particularly emphasizes the identification of key fuel manufacturing parameters, the specification of a fuel performance envelope to inform testing requirements, the use of evaluation models in the fuel qualification process, and the assessment of the experimental data used to develop and validate the empirical models and empirical safety criteria.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Adaptive Discovery and Mixed-Variable Optimization of Next Generation Synthesizable Microelectronic Materials

Design of new microelectronic materials is characterized by several challenges such as high-dimensionality of the atomic structure-composition variable space, formidable cost of directly using high-fidelity simulations for design optimization, dispersity in literature-reported similar materials and synthesis methods, complex physical mechanisms, and mixed qualitative and quantitative design variables that lead to a disjointed design space. Even though machine learning (ML) techniques have been employed to expedite materials innovation, existing methods treat ML and design optimization as two separate processes, failing to resolve the fundamental challenges associated with high dimensionality and mixed-variable complexity. We have developed a ML enhanced mixed-variable material design optimization framework to efficiently extract useful information from existing data in literature and physics-based simulations to guide the autonomous search for optimal materials. Our proposed framework is composed of four computational modules: (1) a natural language processing (NLP) based virtual screening module, (2) classification based concept exploration module, (3) a density functional theory (DFT)-based high-fidelity evaluation model, and (4) a novel latent-variable Gaussian process (LVGP) ML model for mixed-variable problems with uncertainty quantification, which seamlessly integrates with Bayesian Optimization (BO) and achieves superb efficiency through embedded physics-based dimension reduction. Our approach is demonstrated and validated using the testbed of functional materials exhibiting metal-insulation transitions (MITs), with the targeted reversible resistivity changes (∼10^5) near room temperature. At the end of the 30-month project, we have developed a series of new ML techniques using NLP, conditional variational autoencoders, active learning, latent-variable Gaussian processes, integrated with Bayesian optimization. Our project has resulted in new predicted MITs compounds and improved understanding of MITs microscopic mechanisms, which in turn will revolutionize microelectronics science to provide energy-saving solutions. Our research has improved both creativity and efficiency in transforming rare-event discoveries of new functional materials to persistent innovations. In addition to open-sourcing the online MIT database and the classification model, the LVGP open source code has been downloaded more than 15,000 times within two years. More than 40 MIT compounds have been identified and many have been pursued experimentally via collaborators. The research results are published in close to 20 collaborative papers in high-impact journals, such as Chem. Mater., Appl. Phys. Rev., Sci. Rep., among others of design space.

36 MATERIALS SCIENCE↗

Emergent Behaviors and the Limits of Large Language Model Generalization

Over the past couple years, large language models (LLMs) have rapidly grown in size and performance across wide numbers of tasks, leading some to herald them as machine learning models with truly “general” capabilities. We address some of the ways that the language used to describe these models can be both deceptive and a poor framing for making progress towards understanding their capabilities, and also review arguments on to what extent these models can represent and interact with the meaning of language. Emergent behaviors in these highly complex models are unpredictable and research into understanding them is still in early phases, which limits the strength of claims of generality about them. Moreover, while these models demonstrate surprising emergent reasoning capabilities on many tasks, there are still many limitations in generalization that overall high benchmark performance can hide. We review the literature showcasing limitations of LLMs as well as techniques to mitigate or overcome these challenges, while also highlighting how fundamental problems in model evaluation may prevent true claims of generality. We conclude with a practical section on using LLMs on real world problems while appropriately evaluating and validating these tasks to prevent or mitigate the downstream impacts of incorrect results.

97 MATHEMATICS AND COMPUTING↗

Cooperative Agreement To Analyze variabiLity, change and predictabilitY in the earth SysTem (CATALYST)

CATALYST proposes to perform foundational coordinated research in a team-oriented collaborative effort aimed at advancing a robust understanding of modes of Earth system variability and change using models, observations and process studies. The proposed research will address the DOE/BER mission by exploring the limits to predictability, identifying fundamental underlying mechanisms, quantifying interactions among modes of variability, and discovering tipping points in the Earth system to understand the current and future impacts of these phenomena on regional and global climate. Four fundamental gaps are identified in our knowledge of the Earth system: 1) What are the limits to predictability on various timescales? 2) What are the interactions among modes of Earth system variability? 3) How may modes of Earth system variability change in response to changes in external forcing, and what are the tipping points involved with those changes? 4) How are high impact events connected to modes of Earth system variability and how may they change in the future? Related to those gaps in our knowledge, we formulate four research objectives to address those gaps using a combination of Earth system models (ESMs) and machine learning (ML) methods. Research Objective 1 (RO1) addresses the first gap above and proposes to understand modes of variability and their limits of predictability on subseasonal to decadal timescales using ESMs and ML. Research Objective 2 (RO2) addresses the second gap and proposes to use a hierarchy of models to understand relevant processes and feedbacks related to how modes of variability interact with each other. Research Objective 3 (RO3) is designed to study the third gap and proposes to examine the role of external forcings in changes of modes of Earth system variability and their interactions, and the likelihood and predictability of tipping points and irreversible changes. Research Objective 4 (RO4) will address the fourth gap and proposes to use high resolution ESMs, regionally refined models (RRMs), and ML methods to investigate the relationships between high impact events (e.g. flash droughts and precipitation extremes, atmospheric rivers (ARs), tropical cyclones (TCs), storm surge/sea level rise), the synoptic systems that produce them, and their changes related to modes of Earth system variability. The research will involve the use of the Community Earth System Model (CESM), Energy Exascale Earth System Model (E3SM), CMIP multi-model data sets, a hierarchy of simpler models, and numerous observational data sets. In the course of the proposed research, CATALYST will contribute to metrics and diagnostics that will be integrated in Coordinated Model Evaluation Capabilities (CMEC), particularly with regards to the Quasi-biennial Oscillation (QBO) and its interactions with the Madden-Julian Oscillation (MJO), high atmospheric pressure blocking, and new precipitation metrics.

54 ENVIRONMENTAL SCIENCES↗

Multi-Scale Modeling and Prototype Development for Electrochemical CO2 Reduction (CRADA Final Report)

In this CRADA project, Lawrence Livermore National Laboratory, Stanford University, SLAC National Laboratory, and TotalEnergies collaboratively executed a multidisciplinary investigation of electrochemical reduction of CO2 to produce sustainable fuels and chemicals. Overall, the project led to an increased understanding of the fundamental processes involved in CO2 electrolysis, from the atomistic scale to the full electrolyzer device scale, ultimately leading to design guidelines for CO2 electrolyzers that will help in their future commercialization. As the model systems, Ag- and Cu-based catalysts were investigated in various forms depending on the electrochemical platform that was utilized to study the activity, selectivity, and durability towards electrochemical CO2 reduction. By employing experimental, theoretical, and computational techniques, the project team experimentally validated multi-physics models, evaluated the experimental levers that lead to increased electrolyzer reaction selectivity and energy efficiency, and used computational optimization to design higher performance electrodes. The learnings of this project were extensively documented in publicly available peer-reviewed journal publications and conference presentations, which serve as a foundation for further work to build from.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Fuel-Cladding Eutectic Study of Legacy Fast Flux Test Facility (FFTF) MFF HT9/U-10Zr Metallic Fuel

This report presents the first systematic investigation of fuel-cladding eutectic interaction (FCEI) in irradiated HT9/U-10Zr metallic fuel from the Fast Flux Test Facility (FFTF) Materials Fuels Form (MFF) program, using differential scanning calorimetry (DSC) coupled with scanning electron microscopy (SEM) and energy dispersive X-ray spectroscopy (EDS). Two irradiated fuel cross-sections, MNT07H (9.5 at% burnup, x/L = 0.78) and MNT08H (7.0 at% burnup, x/L = 0.93), were subjected to three successive isothermal annealing rounds (R1–R3) at 820°C for 20 minutes each, yielding a cumulative transient duration of one hour. This study directly addresses a recognized gap in the existing FCEI database, which previously lacked irradiated HT9/U-10Zr data at burnup levels above 8 at%. Two principal findings emerge from the study. First, for both samples, FCEI remained spatially confined within the pre-existing fuel-cladding chemical interaction (FCCI) zone boundaries after R3, with no measurable eutectic penetration into unaffected cladding beyond the original FCCI layer. This self-limiting behavior is consistent with historical Fuel Behavior Test Apparatus (FBTA) results and is attributed to the near-eutectic phase composition of the FCCI zone, which rapidly absorb the available eutectic-forming constituents and then stall penetration once the FCCI zone is consumed. A comparison with unirradiated surrogate data further supports this mechanism: whereas a U–34 at.% Fe sample would be expected to show ~176 µm of iron penetration under comparable conditions, the irradiated samples exhibited only ~20 µm, a discrepancy attributed to irradiation-induced interfacial porosity and pre-existing FCCI composition gradients. Second, for MNT08H, FCEI was observed exclusively on the half of the cladding circumference where pre-existing steady-state FCCI was present, with no detectable FCEI on the opposite half. Three hypotheses are proposed to explain this asymmetry: the inhibiting role of a zirconium-rich rind at the fuel-cladding interface; the chemical sequestration of iron by redistributed zirconium within the fuel matrix; and the persistence of fuel-cladding gaps on the FCEI-free half that preclude direct contact. All three hypotheses require further experimental investigation. The results extend the empirical FCEI database into higher-burnup territory and demonstrate the viability of DSC-based testing as a substitute for the no-longer-available FBTA apparatus. Future work will include additional cross-section testing, compilation of the full FCEI dataset, model evaluation, and DSC testing of ternary fuel compositions to broaden the experimental basis for safety assessment of sodium-cooled fast reactor systems.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Search for HH → bbτ⁺τ⁻ Using Run 3 Scouting Data Analyze b-tagging and tau-tagging Performance with Unified Particle Transformer

B-tagging and tau-tagging performances play an important role in the search for the rare event HH → bbτ⁺τ⁻. A transformer-based neural network, Unified Particle Transformer, is applied for both tagging tasks, and Run 3 proton–proton collision scouting data at center-of-mass energy of 13.6 TeV is used. The scouting data stream accepts events at a much higher rate compared to traditional triggers, but stores only the objects reconstructed in the trigger, no low-level detector information. Therefore, existing taggers trained for the offline event reconstruction cannot be used. Analysis of the SoftMax plots, ROC/AUC curves, confusion matrix, accuracy and losses are used to evaluate model performance. Specifically, the tagging efficiency of the signal and misidentification probability across multiple background processes are compared for varying working points. Different training samples with distinct distributions of jet flavors are utilized and related model performances are analyzed. Interpretability methods, such as Integrated Gradients, may further be applied to study the input features’ influence on the model’s decisions, providing insights into potential improvements.

Chen, Blair [Purdue U., West Lafayette; Fermilab]↗

Community Response to Concentrating Solar Power in the San Luis Valley: October 9, 2008 - March 31, 2010

This report is about the social acceptance of utility-scale concentrating solar power (CSP) plants in the San Luis Valley, approximately 200 miles southwest of Denver, Colorado. The research focused on social factors that may facilitate and impede the adoption and implementation of CSP. During the winter of 2008-2009, interviews were conducted with a purposive sample of 25 CSP-related stakeholders inside and outside the Valley. Interviews focused on the perceived advantages and disadvantages of siting a hypothetical 100-MW CSP facility in the Valley, the level of community support and opposition to CSP development, and related issues, such as transmission. State policy recommendations based on the findings include developing education programs for Valley residents, integrating Valley decision makers into an energy-water-land group, providing training for Valley decision makers, offering workforce training, evaluating models of taxation, and forming landholder energy associations. In addition, the SLV could become a laboratory for new approaches to CSP facility and transmission siting decision-making. The author recommends that outside stakeholders address community concerns and engage Valley residents in CSP decisions. Engaging the residents in CSP and transmission decisions, the author says, should take parallel significance with the investment in solar technology.

14 SOLAR ENERGY↗

Event Log / Raw Data

The WFIP3 event log is a curated record spanning 578 days of meteorological phenomena and field observations that complements the campaign’s high-frequency measurements. The log combines manually documented daily weather discussions with automatically derived indicators of key atmospheric processes, providing standardized, publicly available context to support model evaluation, forecast verification, and case-study selection for offshore boundary-layer research.

17 WIND ENERGY↗

Sensitivity of a Subject-specific Ankle Sprain Simulation to Extrinsic Versus Intrinsic Biomechanical Factors

Ankle sprains are the most common musculoskeletal injury in sport and military activity, despite existing prophylactic strategies. The purpose of this report was to develop a probabilistic simulation of lateral ankle sprains during single-limb drop landing, towards accelerating innovation in ankle sprain prevention. A deterministic, subject-specific musculoskeletal model was extended with automation and probabilistic distributions on sprain-related biomechanical factors. Probabilistic simulations were generated using traditional Monte Carlo techniques and the advanced mean value method, a more computationally-efficient approach. Predicted distributions of peak ankle joint rotations, velocities, and moments borne by supporting passive structures agreed favorably with the deterministic model and with reports of real sprain biomechanics. Parameter sensitivities identified that predictions were most strongly influenced by drop height, subtalar joint posture at contact, invertor/evertor co-activation, and passive ankle stiffness. The advanced mean value method predicted confidence bounds comparable to a 1000-trial Monte Carlo simulation, and required only 14 model evaluations and 4-min processing time. The extended probabilistic simulation may be useful to virtually test new prophylactic strategies for ankle sprains, and is made available for open-source use (https://simtk.org/projects/sprain-sim).

60 APPLIED LIFE SCIENCES↗

Statistical Relations among Solid Precipitation, Atmospheric Moisture and Cloud Parameters in the Arctic

Observations collected during cold-season precipitation periods at Utquagvik, Alaska and at the multidisciplinary drifting observatory for the study of Arctic climate (MOSAiC) are used to statistically analyze the relations among the atmospheric water cycle parameters including the columnar supercooled liquid and ice amounts (expressed as liquid-water and ice-water paths, i.e., LWP and IWP), the integrated water vapor (IWV) and the near-surface snowfall rate. Data come from radar and radiometer-based retrievals and from optical precipitation sensors. While the correlation between snowfall rate and LWP is rather weak, correlation coefficients between radar-derived snowfall rate and IWP are high (~0.8), which is explained, in part, by the generally low LWP/IWP ratios during significant precipitation. Correlation coefficients between snowfall rate and IWV are moderate (~0.45). Correlations are generally weaker if snowfall is estimated by optical sensors, which is, in part, due to blowing snow. Correlation coefficients between near-surface temperature and snowfall rates are low (r < 0.3). The results from the Alaska and MOSAiC sites are generally similar. These results are not very sensitive to the amount of time averaging (e.g., 15 min averaging versus daily averages). Observationally based relations among the water cycle parameters are informative about atmospheric moisture conversion processes and can be used for model evaluations.

54 ENVIRONMENTAL SCIENCES↗

Marine Boundary Layer Cloud Boundaries and Phase Estimation Using Airborne Radar and In Situ Measurements During the SOCRATES Campaign over Southern Ocean

The Southern Ocean Clouds, Radiation, Aerosol Transport Experimental Study (SOCRATES) was an aircraft-based campaign (15 January–26 February 2018) that deployed in situ probes and remote sensors to investigate low-level clouds over the Southern Ocean (SO). A novel methodology was developed to identify cloud boundaries and classify cloud phases in single-layer, low-level marine boundary layer (MBL) clouds below 3 km using the HIAPER Cloud Radar (HCR) and in situ measurements. The cloud base and top heights derived from HCR reflectivity, Doppler velocity, and spectrum width measurements agreed well with corresponding lidar-based and in situ estimates of cloud boundaries, with mean differences below 100 m. A liquid water content–reflectivity (LWC-Z) relationship, LWC = 0.70Z0.29, was derived to retrieve the LWC and liquid water path (LWP) from HCR profiles. The cloud phase was classified using HCR measurements, temperature, and LWP, yielding 40.6% liquid, 18.3% mixed-phase, and 5.1% ice samples, along with drizzle (29.1%), rain (3.2%), and snow (3.7%) for drizzling cloud cases. The classification algorithm demonstrates good consistency with established methods. This study provides a framework for the boundary and phase detection of MBL clouds, offering insights into SO cloud microphysics and supporting future efforts in satellite retrievals and climate model evaluation.

MBL clouds over Southern Ocean↗

Grid Cyber-Security Strategy in an Attacker-Defender Model

The progression of cyber-attacks on the cyber-physical system is analyzed by the Probabilistic, Learning Attacker, and Dynamic Defender (PLADD) model. Although our research does apply to all cyber-physical systems, we focus on power grid infrastructure. The PLADD model evaluates the effectiveness of moving target defense (MTD) techniques. We consider the power grid attack scenarios in the AND configurations and OR configurations. In addition, we consider, for the first time ever, power grid attack scenarios involving both AND configurations and OR configurations simultaneously. Cyber-security managers can use the strategy introduced in this manuscript to optimize their defense strategies. Specifically, our research provides insight into when to reset access controls (such as passwords, internet protocol addresses, and session keys), to minimize the probability of a successful attack. Our mathematical proof for the OR configuration of multiple PLADD games shows that it is best if all access controls are reset simultaneously. For the AND configuration, our mathematical proof shows that it is best (in terms of minimizing the attacker's average probability of success) that the resets are equally spaced apart. We introduce a novel concept called hierarchical parallel PLADD system to cover additional attack scenarios that require combinations of AND and OR configurations.

97 MATHEMATICS AND COMPUTING↗

Active Optical Control with Machine Learning: A Proof of Concept for the Vera C. Rubin Observatory

Abstract The Active Optics System of the Vera C. Rubin Observatory (Rubin) uses information provided by four wave front sensors to determine deviations between the reconstructed wave front and the ideal wave front. The observed deviations are used to adjust the control parameters of the optical system to maintain image quality across the 3.°5 field of view. The baseline approach from the project is to obtain amplitudes of the Zernike polynomials describing the distorted wave front from out-of-focus images collected by the wave front sensors. These Zernike amplitudes are related via an “influence matrix” to the control parameters necessary to correct the wave front. In this paper, we use deep-learning methods to extract the control parameters directly from the images captured by the wave front sensors. Our neural net model uses anti-aliasing pooling to boost performance, and a domain-specific loss function to aid learning and generalization. The accuracy of the control parameters derived from our model exceeds Rubin requirements even in the presence of full-moon background levels and mis-centering of reference stars. Although the training process is time consuming, model evaluation requires only a few milliseconds. This low latency should allow for the correction of the optical configuration during the readout and slew interval between successive exposures.

Astronomy & Astrophysics↗