Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Benchmarking machine learning strategies for phase-field problems

Abstract We present a comprehensive benchmarking framework for evaluating machine-learning approaches applied to phase-field problems. This framework focuses on four key analysis areas crucial for assessing the performance of such approaches in a systematic and structured way. Firstly, interpolation tasks are examined to identify trends in prediction accuracy and accumulation of error over simulation time. Secondly, extrapolation tasks are also evaluated according to the same metrics. Thirdly, the relationship between model performance and data requirements is investigated to understand the impact on predictions and robustness of these approaches. Finally, systematic errors are analyzed to identify specific events or inadvertent rare events triggering high errors. Quantitative metrics evaluating the local and global description of the microstructure evolution, along with other scalar metrics representative of phase-field problems, are used across these four analysis areas. This benchmarking framework provides a path to evaluate the effectiveness and limitations of machine-learning strategies applied to phase-field problems, ultimately facilitating their practical application.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Integral analysis of the effect of material dimension and composition on tokamak neutronics *

Abstract The neutronics performance of a tokamak has been identified as an important factor in designing a fusion power plant. The design of the tokamak should not only meet operational parameters such as sufficient tritium breeding, but also safety parameters such as low structural material activation. This paper investigates the impacts of the neutronics metrics for the ARC-class tokamak, a compact tokamak with an immersion blanket, by perturbing the first five layers of structural material—first wall, inner vacuum vessel, coolant salt channel, neutron multiplier, and outer vacuum vessel. The goal of this work is to provide insight into shaping and scaling the flux on each layer to obtain optimized operational and safety metrics through quantification of the responses from each perturbation. Results show that increased first wall thickness can increase the tritium breeding ratio (TBR) in specific configurations with high 6 Li enrichments and that vacuum vessels decrease TBR for low- 6 Li enrichment configurations. It was also found that the neutron multiplier can either increase or decrease TBR depending on the configuration. The response of metrics to the change in layer thickness and enrichment also varies depending on the vacuum vessel material. The integral impacts of 6 Li enrichment, layer thicknesses, and vacuum vessel material choice are investigated and presented in this paper.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Multi-device analysis of energy loss duration and pellet penetration with implications for shattered pellet injection in ITER

A robust disruption mitigation system (DMS) requires accurate characterization of key disruption timescales, one of the most notable being the thermal quench (TQ). Recent modeling of shattered pellet injection (SPI) into ITER plasmas, using JOREK and INDEX, suggests long TQ durations (6–10 ms) and slow cold front propagation due to the large plasma size. If validated, these predictions would have an impact on the desired pellet parameters and mitigation strategies for the ITER DMS. To resolve these questions, a database of SPI experiments from several small-to-large sized devices (J-TEXT, KSTAR, AUG, DIII-D, and JET) has been compiled under the auspices of the International Tokamak Physics Activity MHD, disruptions, and control topical group. Analysis of the energy loss duration (proxy for the TQ duration) with machine size is presented for both mixed neon/deuterium (Ne/D) SPI and pure deuterium (D) SPI. Several metrics for the energy loss onset (e.g. soft x-ray signal drop, I p dip, and radiation flash) were considered as the conventional metric, electron cyclotron emission, is often cut-off during SPI. Several scalings with different onset metrics showed an increase in energy loss duration with machine size. The energy loss duration was additionally shown to be a function of the ratio between the number of SPI neon atoms injected and the stored energy. Analysis of the pellet shard position relative to the cold front found that in larger devices, pellets are typically found inboard of the q = 2 surface at the energy loss onset. Lastly, the delay between the pellet shards hitting the q = 2 surface and the energy loss onset was additionally found to increase with machine size. This suggests that the pellet shards in large devices will penetrate faster and further than the cooling front.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Balancing performance of active magnetic regenerators: a comprehensive experimental study of aspect ratio, particle size, and operating conditions

Abstract Effective and, at the same time, efficient active magnetic regenerator (AMR) performance requires balanced geometry and operating conditions. Here the influence of regenerator shape, magnetocaloric material size, operating frequency, and utilization on the performance of gadolinium packed-particle bed AMRs is demonstrated experimentally. Various metrics are applied to assess effectiveness and efficiency. Observed temperature spans and cooling powers across a wide range of operating conditions are used to evaluate system performance and estimate exergetic cooling power and exergetic power quotient. A new metric combining exergetic cooling power and pump power provides an estimate of the maximum achievable second law efficiency. Five regenerator geometries with equal volumes and the aspect ratio from 1.0 to 3.8, and four different ranges of Gd spherical particles between 182 and 354 µ m, are investigated. Improvements in system performance are demonstrated by a boost in specific cooling power of gadolinium from 0.85 to 1.16 W g −1 and maximum temperature span from 8.9 to 15.1 K. The optimum exergetic cooling power is observed for 1.37 utilization and 3 Hz operating frequency, exergetic power quotient exhibits a maximum at the same utilization but at 2 Hz frequency, while the highest efficiency is recorded at 1 Hz and utilization of 0.5, demonstrating that multiple performance metrics must be balanced to achieve regenerator design meeting all performance targets.

42 ENGINEERING↗
DOE OSTI2023

Data efficiency and extrapolation trends in neural network interatomic potentials

Abstract Recently, key architectural advances have been proposed for neural network interatomic potentials (NNIPs), such as incorporating message-passing networks, equivariance, or many-body expansion terms. Although modern NNIP models exhibit small differences in test accuracy, this metric is still considered the main target when developing new NNIP architectures. In this work, we show how architectural and optimization choices influence the generalization of NNIPs, revealing trends in molecular dynamics (MD) stability, data efficiency, and loss landscapes. Using the 3BPA dataset, we uncover trends in NNIP errors and robustness to noise, showing these metrics are insufficient to predict MD stability in the high-accuracy regime. With a large-scale study on NequIP, MACE, and their optimizers, we show that our metric of loss entropy predicts out-of-distribution error and data efficiency despite being computed only on the training set. This work provides a deep learning justification for probing extrapolation and can inform the development of next-generation NNIPs.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Surrogate modelling the Baryonic Universe II: On forward modelling the colours of individual and populations of galaxies

ABSTRACT Among the properties shaping the light of a galaxy, the star formation history (SFH) is one of the most challenging to model due to the variety of correlated physical processes regulating star formation. In this work, we leverage the stellar population synthesis model fsps, together with SFHs predicted by the hydrodynamical simulation IllustrisTNG and the empirical model universemachine, to study the impact of star formation variability on galaxy colours. We start by introducing a model-independent metric to quantify the burstiness of a galaxy formation model, and we use this metric to demonstrate that universemachine predicts SFHs with more burstiness relative to IllustrisTNG. Using this metric and principal component analysis, we construct families of SFH models with adjustable variability, and we show that the precision of broad-band optical and near-infrared colours degrades as the level of unresolved short-term variability increases. We use the same technique to demonstrate that variability in metallicity and dust attenuation presents a practically negligible impact on colours relative to star formation variability. We additionally provide a model-independent fitting function capturing how the level of unresolved star formation variability translates into imprecision in predictions for galaxy colours; our fitting function can be used to determine the minimal SFH model that reproduces colours with some target precision. Finally, we show that modelling the colours of individual galaxies with per cent-level precision demands resorting to complex SFH models, while producing precise colours for galaxy populations can be achieved using models with just a few degrees of freedom.

Chaves-Montero, Jonás (ORCID:0000000295534261)↗
DOE OSTI2022

The merger fraction of post-starburst galaxies in UNIONS

ABSTRACT Post-starburst galaxies (PSBs) are defined as having experienced a recent burst of star formation, followed by a prompt truncation in further activity. Identifying the mechanism(s) causing a galaxy to experience a post-starburst phase therefore provides integral insight into the causes of rapid quenching. Galaxy mergers have long been proposed as a possible post-starburst trigger. Effectively testing this hypothesis requires a large spectroscopic galaxy survey to identify the rare PSBs as well as high-quality imaging and robust morphology metrics to identify mergers. We bring together these critical elements by selecting PSBs from the overlap of the Sloan Digital Sky Survey and the Canada–France Imaging Survey and applying a suite of classification methods: non-parametric morphology metrics such as asymmetry and Gini-M20, a convolutional neural network trained to identify post-merger galaxies, and visual classification. This work is therefore the largest and most comprehensive assessment of the merger fraction of PSBs to date. We find that the merger fraction of PSBs ranges from 19 per cent to 42 per cent depending on the merger identification method and details of the PSB sample selection. These merger fractions represent an excess of 3–46× relative to non-PSB control samples. Our results demonstrate that mergers play a significant role in generating PSBs, but that other mechanisms are also required. However, applying our merger identification metrics to known post-mergers in the IllustrisTNG simulation shows that 70 per cent of recent post-mergers (≲200 Myr) would not be detected. Thus, we cannot exclude the possibility that nearly all PSBs have undergone a merger in their recent past.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

CarcSeq Measurement of Rat Mammary Cancer Driver Mutations and Relation to Spontaneous Mammary Neoplasia

Abstract The ability to deduce carcinogenic potential from subchronic, repeat dose rodent studies would constitute a major advance in chemical safety assessment and drug development. This study investigated an error-corrected NGS method (CarcSeq) for quantifying cancer driver mutations (CDMs) and deriving a metric of clonal expansion predictive of future neoplastic potential. CarcSeq was designed to interrogate subsets of amplicons encompassing hotspot CDMs applicable to a variety of cancers. Previously, normal human breast DNA was analyzed by CarcSeq and metrics based on mammary-specific CDMs were correlated with tissue donor age, a surrogate of breast cancer risk. Here we report development of parallel methodologies for rat. The utility of the rat CarcSeq method for predicting neoplastic potential was investigated by analyzing mammary tissue of 16-week-old untreated rats with known differences in spontaneous mammary neoplasia (Fischer 344, Wistar Han, and Sprague Dawley). Hundreds of mutants with mutant fractions ≥ 10−4 were quantified in each strain, most were recurrent mutations, and 42.5% of the nonsynonymous mutations have human homologs. Mutants in the mammary-specific target of the most tumor-sensitive strain (Sprague Dawley) showed the greatest nonsynonymous/synonymous mutation ratio, indicative of positive selection consistent with clonal expansion. For the mammary-specific target (Hras, Pik3ca, and Tp53 amplicons), median absolute deviation correlated with percentages of rats that develop spontaneous mammary neoplasia at 104 weeks (Pearson r = 1.0000, 1-tailed p = .0010). Therefore, this study produced evidence CarcSeq analysis of spontaneously occurring CDMs can be used to derive an early metric of clonal expansion relatable to long-term neoplastic outcome.

McKim, Karen L.↗
DOE OSTI2020

Numerical relativity in spherical coordinates: A new dynamical spacetime and general relativistic MHD evolution framework for the Einstein Toolkit

We present SphericalNR, a new framework for the publicly available Einstein Toolkit that numerically solves the Einstein field equations coupled to the equations of general relativistic magnetohydrodynamic (GRMHD) in a 3+1 split of spacetime in spherical coordinates without symmetry assumptions. The spacetime evolution is performed using reference-metric versions of either the Baumgarte-Shapiro-Shibata-Nakamura equations or the fully covariant and conformal Z4 system with constraint damping. We have developed a reference-metric version of the Valencia formulation of GRMHD with a vector potential method, guaranteeing the absence of magnetic monopoles during the evolution. In our framework, every dynamical field (both spacetime and matter) is evolved using its components in an orthonormal basis with respect to the spherical reference metric. Furthermore, all geometric information about the spherical coordinate system is encoded in source terms appearing in the evolution equations. This allows for the straightforward extension of Cartesian high-resolution shock-capturing finite volume codes to use spherical coordinates with our framework. To this end, we have adapted GRHydro, a Cartesian finite volume GRMHD code already available in the Einstein Toolkit, to use spherical coordinates. We present the full evolution equations of the framework, as well as details of its implementation in the Einstein Toolkit. Finally, we validate SphericalNR by demonstrating it passes a variety of challenging code tests in static and dynamical spacetimes.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

MeV-scale performance of water-based and pure liquid scintillator detectors

This paper presents studies of the performance of water-based liquid scintillators (WbLS) in both 1-kt and 50-kt detectors. In this work, performance is evaluated in comparison to both pure water Cherenkov detectors and a nominal model for pure scintillator detectors. Performance metrics include energy, vertex, and angular resolution, along with a metric for the ability to separate the Cherenkov from the scintillation signal as being representative of various particle identification capabilities that depend on the Cherenkov/scintillation ratio. We also modify the time profile of scintillation light to study the same performance metrics as a function of rise and decay time. We go on to interpret these results in terms of their impact on certain physics goals, such as solar neutrinos and the search for Majorana neutrinos. This work supports and validates previous results, and the assumptions made therein, and serves as a significant stepping stone to complete detector design studies by using a more detailed detector model and full reconstruction, with a primarily data-driven optical model, and fewer model assumptions. With this model, a high-coverage 50-kt detector achieves better than 10 (1)% precision on the flux of neutrinos from the Carbon-Nitrogen-Oxygen cycle with a water-based liquid scintillator (pure LS) target in five years of data taking. A 1-kt LS detector, with a conservative 50% fiducial volume of 0.5 kt, can achieve better than 5% detection. A liquid scintillator detector has sensitivity into the normal hierarchy region for Majorana neutrinos with half-life sensitivity of $T^{0νββ}_{1/2}$ > 1.4 × 10 28 years at 90% C.L. for 10 years of data taking with a Te-loaded target.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Accurate relativistic observables from postprocessing light cone catalogs

We introduce and study a new scheme to construct relativistic observables from postprocessing light cone data. This construction is based on a novel approach, LC-metric, which takes general light cone or snapshot output generated by arbitrary N-body simulations or emulations and solves the linearized Einstein equations to determine the spacetime metric on the light cone. We find that this scheme is able to determine the metric to high precision, and subsequently generate accurate mock cosmological observations sensitive to effects such as post-Born lensing and nonlinear integrated Sachs-Wolfe contributions. By comparing to conventional methods in quantifying those general relativistic effects, we show that this scheme is able to accurately construct the lensing convergence signal. We also find that the accuracy of this method in quantifying the integrated Sachs-Wolfe effects in the highly nonlinear regime outperforms conventional methods by an order of magnitude. Furthermore, this scheme opens a new path for exploring and modeling higher-order and nonlinear general relativistic contributions to cosmological observables, including mock observations of gravitational lensing and the moving lens and Rees-Sciama effects.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Self-consistent optimization of the z expansion for B -meson decays

We discuss the self-consistency imposed by the analyticity of regular parts of form factors, appearing in the z expansion for semileptonic B -meson decays, when fitted in different kinematic regions. Relying on the uniqueness of functions defined by analytic continuation, we propose four metrics which measure the departure from the ideal analytic self-consistency. We illustrate the process using Belle data for B → D ℓ ν ℓ with the two kinematic regions chosen as the five low- z and the five high- z bins. For this specific example, the metrics provide consistent indications that some choices (order of truncation, Boyd-Grinstein-Lebed or Bourrely-Caprini-Lellouch) made in the form of the z expansion can be optimized. However, other choices ( z origin, location of isolated poles and threshold constraints) appear to have very little effect on these metrics. On the other hand, changing the kinetic regions affects the results and should also be considered in the optimization process. We briefly discuss the implication for optimization of the z expansion for nucleon form factors relevant for neutrino oscillation experiments. Published by the American Physical Society 2024

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Physics-informed transformation toward improving the machine-learned NLTE models of ICF simulations

The integration of machine-learning techniques into inertial confinement fusion (ICF) simulations has emerged as a powerful approach for enhancing computational efficiency. By replacing the costly nonlocal thermodynamic equilibrium (NLTE) model with machine-learning models, significant reductions in calculation time have been achieved. However, determining how to optimize machine-learning-based NLTE models in order to match ICF simulation dynamics remains challenging, underscoring the need for physically relevant error metrics and strategies to enhance model accuracy with respect to these metrics. Thus, we propose novel physics-informed transformations designed to emphasize energy transport, use these transformations to establish new error metrics, and demonstrate that they yield smaller errors within reduced principal-component spaces compared to conventional transformations. Published by the American Physical Society 2025

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Quantifying Quantum Chaos through Microcanonical Distributions of Entanglement

A characteristic feature of “quantum chaotic” systems is that their eigenspectra and eigenstates display universal statistical properties described by random matrix theory (RMT). However, eigenstates of local systems also encode structure beyond RMT. To capture this feature, we introduce a framework that allows us to compare the properties of eigenstates in local systems with those of pure random states. In particular, our framework defines a notion of distance between quantum state ensembles that utilizes the Kullback-Leibler divergence to compare the microcanonical distribution of entanglement entropy (EE) of eigenstates with a reference RMT distribution generated by pure random states (with appropriate constraints). This notion gives rise to a quantitative metric for quantum chaos that not only accounts for averages of the distributions but also higher moments. The differences in moments are compared on a highly resolved scale set by the standard deviation of the RMT distribution, which is exponentially small in system size. As a result, the metric can distinguish between chaotic and integrable behaviors and, in addition, quantify and compare the of chaos (in terms of proximity to RMT behavior) between two systems that are assumed to be chaotic. We implement our framework in local, minimally structured, Floquet random circuits, as well as a canonical family of many-body Hamiltonians, the mixed-field Ising model (MFIM). Importantly, for Hamiltonian systems, we find that the reference random distribution must be appropriately constrained to incorporate the effect of energy conservation in order to describe the ensemble properties of midspectrum eigenstates. The metric captures deviations from RMT across all models and parameters, including those that have been previously identified as strongly chaotic, and for which other diagnostics of chaos such as level spacing statistics look strongly thermal. In Floquet circuits, the dominant source of deviations is the second moment of the distribution, and this persists for all system sizes. For the MFIM, we find significant variation of the KL divergence in parameter space. Notably, we find a small region where deviations from RMT are minimized, suggesting that “maximally chaotic” Hamiltonians may exist in fine-tuned pockets of parameter space. Published by the American Physical Society 2024

Physics↗
DOE OSTI2021

CERES : a cryo-EM re-refinement system for continuous improvement of deposited models

The field of electron cryomicroscopy (cryo-EM) has advanced quickly in recent years as the result of numerous technological and methodological developments. This has led to an increase in the number of atomic structures determined using this method. Recently, several tools for the analysis of cryo-EM data and models have been developed within the Phenix software package, such as phenix.real_space_refine for the refinement of atomic models against real-space maps. Also, new validation metrics have been developed for low-resolution cryo-EM models. To understand the quality of deposited cryo-EM structures and how they might be improved, models deposited in the Protein Data Bank that have map resolutions of better than 5 Å were automatically re-refined using current versions of Phenix tools. The results are available on a publicly accessible web page (https://cci.lbl.gov/ceres). The implementation of a Cryo-EM Re-refinement System ( CERES ) for the improvement of models deposited in the wwPDB, and the results of the re-refinements, are described. Based on these results, contents are proposed for a `cryo-EM Table 1', which summarizes experimental details and validation metrics in a similar way to `Table 1' in crystallography. The consistent use of robust metrics for the evaluation of cryo-EM models and data should accompany every structure deposition and be reported in scientific publications.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Q -score as a reliability measure for protein, nucleic acid and small-molecule atomic coordinate models derived from 3DEM maps

Atomic coordinate models are important for the interpretation of 3D maps produced with cryoEM and cryoET (3D electron microscopy; 3DEM). In addition to visual inspection of such maps and models, quantitative metrics can inform about the reliability of the atomic coordinates, in particular how well the model is supported by the experimentally determined 3DEM map. A recently introduced metric, Q-score, was shown to correlate well with the reported resolution of the map for well fitted models. Here, we present new statistical analyses of Q-score based on its application to ∼10 000 maps and models archived in the EMDB (Electron Microscopy Data Bank) and PDB (Protein Data Bank). Further, we introduce two new metrics based on Q-score to represent each map and model relative to all entries in the EMDB and those with similar resolution. We explore through illustrative examples of proteins, nucleic acids and small molecules how Q-scores can indicate whether the atomic coordinates are well fitted to 3DEM maps and also whether some parts of a map may be poorly resolved due to factors such as molecular flexibility, radiation damage and/or conformational heterogeneity. These examples and statistical analyses provide a basis for how Q-scores can be interpreted effectively in order to evaluate 3DEM maps and atomic coordinate models prior to publication and archiving.

B factors↗
DOE OSTI2024

How Much Reserve Fuel: Quantifying the Maximal Energy Cost of System Disturbances

Motivated by the design question of additional fuel needed to complete a task in an uncertain environment, this paper introduces metrics to quantify the maximal additional energy used by a control system in the presence of bounded disturbances, compared to a nominal, disturbance-free system. In particular, we consider the task of finite-time stabilization for a linear, time-invariant system. We compare the nominal energy required to achieve this task in the disturbance-free system to the worst-case energy over all feasible disturbances. Solving for the worst-case energy over all disturbances first leads to an optimal control problem with a least-squares solution, and then an infinite-dimensional optimization problem where we derive an upper bound on the solution. The comparison of energies is accomplished using additive and multiplicative metrics, for which we derive bounds. Simulation examples on an ADMIRE fighter jet model demonstrate the practicability of these metrics, and their variation with the distance of the initial condition from the origin and the task completion time.

koopman operator, resilience↗
DOE OSTI2025

Towards Automatically Matching Security Advisories to CPEs: String Similarity-based Vendor Matching

When a vulnerability is reported by the National Vulnerability Database (NVD), affected products are listed in the structured Common Platform Enumeration (CPE) format. Unfortunately, if the vulnerability is in a software library (e.g., Log4j), it will not include CPEs for each product containing that library. In these cases, security operators need to manually read the vendor's or third-party security advisories to see if their product is affected. However, these advisories do not report affected products in a structured format, which prevents automated processing, This paper makes the first effort towards automatically constructing structured CPEs for the vulnerable products in a non-NVD security advisory from the unstructured data in the advisory. Since this is a very challenging problem, this paper specifically focuses on the initial but key step of matching the un-structured vendor names in security advisories to the structured vendor representations in the standard CPE format. We explore the feasibility of using string similarity to solve the problem. The basic idea is to compare a vendor name from the non-NVD advisory with each vendor in the official CPE dictionary. The CPE vendor with the highest similarity score to the advisory's vendor will be considered as the match. We first conduct an experimental, comparative study of multiple mainstream string similarity metrics for this matching problem. To improve the performance, we then design a new string similarity metric that is adapted from an existing metric by weighing different tokens in the advisory's vendor name differently.

McClanahan, Kylie↗