SEARCH · Engineering Papers

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Benchmarking graph neural networks for materials chemistry

Graph neural networks (GNNs) have received intense interest as a rapidly expanding class of machine learning models remarkably well-suited for materials applications. To date, a number of successful GNNs have been proposed and demonstrated for systems ranging from crystal stability to electronic property prediction and to surface chemistry and heterogeneous catalysis. However, a consistent benchmark of these models remains lacking, hindering the development and consistent evaluation of new models in the materials field. Here, we present a workflow and testing platform, MatDeepLearn, for quickly and reproducibly assessing and comparing GNNs and other machine learning models. We use this platform to optimize and evaluate a selection of top performing GNNs on several representative datasets in computational materials chemistry. From our investigations we note the importance of hyperparameter selection and find roughly similar performances for the top models once optimized. We identify several strengths in GNNs over conventional models in cases with compositionally diverse datasets and in its overall flexibility with respect to inputs, due to learned rather than defined representations. Meanwhile several weaknesses of GNNs are also observed including high data requirements, and suggestions for further improvement for applications in materials chemistry are discussed.

36 MATERIALS SCIENCE↗

NASA NTRS2008

Experimental Evaluation of a Planning Language Suitable for Formal Verification

The marriage of model checking and planning faces two seemingly diverging alternatives: the need for a planning language expressive enough to capture the complexity of real-life applications, as opposed to a language simple, yet robust enough to be amenable to exhaustive verification and validation techniques. In an attempt to reconcile these differences, we have designed an abstract plan description language, ANMLite, inspired from the Action Notation Modeling Language (ANML) [17]. We present the basic concepts of the ANMLite language as well as an automatic translator from ANMLite to the model checker SAL (Symbolic Analysis Laboratory) [7]. We discuss various aspects of specifying a plan in terms of constraints and explore the implications of choosing a robust logic behind the specification of constraints, rather than simply propose a new planning language. Additionally, we provide an initial assessment of the efficiency of model checking to search for solutions of planning problems. To this end, we design a basic test benchmark and study the scalability of the generated SAL models in terms of plan complexity.

Butler, Rick W.↗

DOE OSTI2020

Modeling of fluid injection-induced fault reactivation using coupled fluid flow and mechanical interface model

The present study is aimed at developing a numerical model to reproduce coupled hydro-mechanical processes associated with fault reactivation by fluid injection in low permeability rock, as part of the DECOVALEX-2019 project Task B. We proposed a modeling approach for simulating the processes using the TOUGH-FLAC simulator, and modeled a fault reactivation experiment conducted at Mont Terri Rock Laboratory in Switzerland. The first step of the study involved benchmark calculations considering a simplified fault plane and geometry. Fluid flow along a fault was modeled using elements of aperture-sized thickness on the basis of Darcy's law and the cubic law in TOUGH2, whereas the mechanical behavior of a single fault was represented by zero-thickness interface elements in FLAC3D upon which a slip and/or separation is allowed. A methodology to connect a TOUGH2 volume element to a FLAC3D interface element was developed for handling the hydro-mechanical interactions on the fault during fluid injection. Two different fault models for describing the evolutions of hydraulic aperture by elastic fracture opening and failure-induced aperture increase were considered in the benchmark calculations. In the coupling process, the changes in geometrical features and hydrological properties induced by mechanical deformation were continuously updated. The transient responses of the fault and host rock to stepwise pressurization were examined during the simulation. The hydro-mechanical behavior, including the injection flow rate, pressure distribution around the borehole, stress conditions, and displacements in normal and shear directions were monitored in the surrounding rock and along the fault. The results of benchmark calculations suggest that the developed model reasonably represents the hydro-mechanical behavior of a fault and the surrounding rock. This modeling approach was applied to the fault reactivation experiment of the Mont Terri Rock Laboratory. In this interpretive modeling, a parametric study was conducted to examine the effects of input parameters regarding in situ stress and fault properties on the hydro-mechanical responses of the fault to water injection. Then, an optimal parameter set to reproduce the field experiment results was chosen by trial-and-error. The injection flow rate and pressure response during fault reactivation closely matched those obtained at the site, which indicates the capability of the model to appropriately capture the progressive pathway evolution during fault reactivation tests at the site. Finally, the anchor displacements were overestimated by the model, but a fair agreement was obtained in terms of the order of magnitude and the variation tendency.

58 GEOSCIENCES↗

DOE OSTI2023

Short-Term Transcriptomic Points of Departure Are Consistent with Chronic Points of Departure for Three Organophosphate Pesticides across Mouse and Fathead Minnow

New approach methods (NAMs) can reduce the need for chronic animal studies. Here, we apply benchmark dose (concentration) (BMD(C))–response modeling to transcriptomic changes in the liver of mice and in fathead minnow larvae after short-term exposures (7 days and 1 day, respectively) to several dose/concentrations of three organophosphate pesticides (OPPs): fenthion, methidathion, and parathion. The mouse liver transcriptional points of departure (TPODs) for fenthion, methidathion, and parathion were 0.009, 0.093, and 0.046 mg/Kg-bw/day, while the fathead minnow larva TPODs were 0.007, 0.115, and 0.046 mg/L, respectively. The TPODs were consistent across both species and reflected the relative potencies from traditional chronic toxicity studies with fenthion identified as the most potent. Moreover, the mouse liver TPODs were more sensitive than or within a 10-fold difference from the chronic apical points of departure (APODs) for mammals, while the fathead minnow larva TPODs were within an 18-fold difference from the chronic APODs for fish species. Short-term exposure to OPPs significantly impacted acetylcholinesterase mRNA abundance (FDR p-value <0.05, |fold change| ≥2) and canonical pathways (IPA, p-value <0.05) associated with organism death and neurological/immune dysfunctions, indicating the conservation of key events related to OPP toxicity. Together, these results build confidence in using short-term, molecular-based assays for the characterization of chemical toxicity and risk, thereby reducing reliance on chronic animal studies.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2021

Reduced Complexity Model Intercomparison Project Phase 2: Synthesizing Earth System Knowledge for Probabilistic Climate Projections

Over the last decades, climate science has evolved rapidly across multiple expert domains. Our best tools to capture state-of-the-art knowledge in an internally self-consistent modelling framework are the increasingly complex fully coupled Earth System Models (ESMs). However, computational limitations and the structural rigidity of ESMs mean that the full range of uncertainties across multiple domains are difficult to capture with ESMs alone. The tools of choice are instead more computationally efficient reduced complexity models (RCMs), which are structurally flexible and can span the response dynamics across a range of domain-specific models and ESM experiments. Here we present Phase 2 of the Reduced Complexity Model Intercomparison Project (RCMIP Phase 2), the first comprehensive intercomparison of RCMs that are probabilistically calibrated with key benchmark ranges from specialized research communities. Unsurprisingly, but crucially, we find that models which have been constrained to reflect the key benchmarks better reflect the key benchmarks. Under the low-emissions SSP1-1.9 scenario, across the RCMs, median peak warming projections range from 1.3 to 1.7°C (relative to 1850-1900, using an observationally-based historical warming estimate of 0.8°C between 1850-1900 and 1995-2014). Further developing methodologies to constrain these projection uncertainties seems paramount given the international community's goal to contain warming to below 1.5°C above pre-industrial in the long-term. Our findings suggest that users of RCMs should carefully evaluate their RCM, specifically its skill against key benchmarks and consider the need to include projections benchmarks either from ESM results or other assessments to reduce divergence in future projections.

Climate↗

DOE OSTI2025

Orthorhombic Structures as Inorganic Halide Perovskite Models for High-Throughput DFT Investigations

Although high-symmetry Pm-3m space group cubic models are computationally efficient for high-throughput density functional theory (DFT) calculations of inorganic ternary (ABX 3 ) halide perovskites (HPs), they frequently predict band gaps (E g ) that disagree with experiment. Conversely, while low-symmetry cubic polymorphous networks (PN) comprised of 160 to 320 atoms incur significantly greater computational cost, they predict E g ’s that are more closely aligned with experiment. Here, in this study, we compare the DFT total energies and E g ’s predicted by four high-symmetry structure models (Pnma orthorhombic, R3m trigonal, P4/mbm tetragonal, and Pm-3m cubic) to cubic PNs for 5 experimentally characterized ternary HPs and find that the orthorhombic model computes Eg’s with the smallest MAD of 0.23 eV relative to the PNs. Pair distribution functions and DFT-computed total energies show that octahedral tilting, which is present in the 20-atom orthorhombic and 160-atom cubic PN models but not in the 5-atom cubic models, stabilizes all 5 compositions in our benchmarking set. We also find that imposing PN constraints when generating and optimizing these orthorhombic structures by fixing the unit cell lattice vectors and displacing the atoms prior to ionic relaxation with DFT, which we call the orthorhombic surrogate model (OSM), lowers the MAD of E g predictions to 0.09 eV. Our OSM predicts the PN band gaps of an additional 95 theoretical inorganic ternary HPs with MAD of 0.08 eV, supporting its usage in high-throughput DFT investigations to closely estimate PN band gaps with much less computational expense.

14 SOLAR ENERGY↗

DOE OSTI2020

Predicting metal–metal interactions. I. The influence of strain on nanoparticle and metal adlayer stabilities

Strain-engineering of bimetallic nanomaterials is an important design strategy for developing new catalysts. In this paper, we introduce an approach for including strain effects into a recently introduced, density functional theory (DFT)-based alloy stability model. The model predicts adsorption site stabilities in nanoparticles and connects these site stabilities with catalytic reactivity and selectivity. Strain-based dependencies will increase the model’s accuracy for nanoparticles affected by finite-size effects. In addition to the stability of small nanoparticles, strain also influences the heat of adsorption of epitaxially grown metal-on-metal adlayers. In this respect, we successfully benchmark the strain-including alloy stability model with previous experimentally determined trends in the heats of adsorption of Au and Cu adlayers on Pt (111). For these systems, our model predicts stronger bimetallic interactions in the first monolayer than monometallic interactions in the second monolayer. We explicitly quantify the interplay between destabilizing strain effects and the energy gained by forming new metal–metal bonds. While tensile strain in the first Cu monolayer significantly destabilizes the adsorption strength, compressive strain in the first Au monolayer has a minimal impact on the heat of adsorption. Hence, this study introduces and, by comparison with previous experiments, validates an efficient DFT-based approach for strain-engineering the stability, and, in turn, the catalytic performance, of active sites in bimetallic alloys with atomic level resolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

Rapid Gamma Simulations of TRISO Fuel Elements

As energy demand rises, nuclear energy, particularly from reactors that use tristructural isotropic (TRISO) fuels, has gained attention due to the fuel’s enhanced resistance to radiation damage and high temperatures. This report investigates the modeling capabilities of the Gamma Detector Response and Analysis Software (GADRAS) for TRISO fuels, focusing on the gamma signatures of TRISO particles, which have not been extensively explored. Using the Monte Carlo N-Particle (MCNP) code as a benchmark, we developed both homogeneous and heterogeneous models of TRISO pebbles to analyze gamma spectra. Our findings reveal that the homogeneous and heterogeneous models produced different gamma signatures. Additionally, the GADRAS heterogeneous model significantly reduces computation times compared to MCNP, enabling effective modeling of gamma signatures for safeguards applications. This advancement is essential for the International Atomic Energy Agency (IAEA) in detecting anomalies and potential smuggling attempts in TRISO reactor fuel elements.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2021

Evaluation and improvement of the parameterization of aerosol hygroscopicity in global climate models using in-situ surface measurements (Final Report)

Aerosols are tiny particles suspended on the atmosphere that can interact with incoming solar radiation and affect the Earth radiative budget. They do so by scattering and absorbing solar radiation, and these properties vary depending on the aerosol size and chemical composition. Moreover, by taking up water from the surrounding air, hygroscopic aerosol particles will grow in size and change their chemical composition, thus modifying their optical properties (scattering and absorption) and their final impact on radiative forcing calculations. An accurate knowledge of aerosol hygroscopicity is crucial for estimating the net radiative impact of aerosols. We took a three-pronged approach to improve our understanding of aerosol hygroscopicity and how it is implemented in Earth system models. In the first part of our project, we developed a benchmark dataset from existing aerosol hygroscopic growth measurements made by tandem nephelometer humidogram systems. We analyzed, using a standardized methodology, observations from 26 in-situ stations around the globe. Measurement data was collected from multiple data providers, reviewed and harmonized to create a consistent dataset of the scattering enhancement factor due to aerosol water uptake. This dataset is archived in several publicly available databases for use by interested researchers. In the second part of the project, we used the benchmark hygroscopicity dataset to perform a global study on aerosol hygroscopicity and aerosol optical properties. Measurements show a global picture of scattering enhancement with larger values for Arctic and marine sites and lower for urban and desert sites. We assessed the RH dependence of aerosol radiative forcing and showed that the overall effect of aerosol hygroscopicity on DARF is an increase in the absolute forcing effect (negative sign) by a factor of up to 4 compared to dry conditions (RH<40%). Finally, we explored using aerosol single scattering albedo (SSA) and scattering Angstrom exponent (SAE) as possible proxies for aerosol hygroscopicity. SSA showed more promise as a surrogate for the scattering enhancement factor than SAE, but neither was ideal. In the third part of the project we evaluated the output of ten Earth system models (ESMs) against the benchmark hygroscopicity dataset. ESMs utilize various schemes for aerosol hygroscopicity which had not been previously tested against observations on a global scale. We found that ESMs currently overestimate scattering enhancement due to hygroscopic growth. Model parameterizations of hygroscopicity and model chemistry are two main factors driving the observed diversity in hygroscopicity simulations among the models. In addition, our study makes several suggestions for modelers, including improving the parameterizations of organic and sea salt aerosol hygroscopicity. Future hygroscopicity model evaluation experiments should include the model data related to particles size which was not available for our study.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Prediction of plant complex traits via integration of multi-omics data

The formation of complex traits is the consequence of genotype and activities at multiple molecular levels. However, connecting genotypes and these activities to complex traits remains challenging. Here, we investigate whether integrating genomic, transcriptomic, and methylomic data can improve prediction for six Arabidopsis traits. We find that transcriptome- and methylome-based models have performances comparable to those of genome-based models. However, models built for flowering time using different omics data identify different benchmark genes. Nine additional genes identified as important for flowering time from our models are experimentally validated as regulating flowering. Gene contributions to flowering time prediction are accession-dependent and distinct genes contribute to trait prediction in different genotypes. Models integrating multi-omics data perform best and reveal known and additional gene interactions, extending knowledge about existing regulatory networks underlying flowering time determination. These results demonstrate the feasibility of revealing molecular mechanisms underlying complex traits through multi-omics data integration.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS1983

Evaluation of Inelastic Constitutive Models for Nonlinear Structural Analysis

The influence of inelastic material models on computed stress-strain states, and therefore predicted lives, was studied for thermomechanically loaded structures. Nonlinear structural analyses were performed on a fatigue specimen which was subjected to thermal cycling in fluidized beds and on a mechanically load cycled benchmark notch specimen. Four incremental plasticity creep models (isotropic, kinematic, combined isotropic-kinematic, combined plus transient creep) were exercised. Of the plasticity models, kinematic hardening gave results most consistent with experimental observations. Life predictions using the computed strain histories at the critical location with a Strainrange Partitioning approach considerably overpredicted the crack initiation life of the thermal fatigue specimen.

Kaufman, A.↗

NASA NTRS2001

NASTRAN DMAP Fuzzy Structures Analysis: Summary of Research

The main proposed tasks of Cooperative Agreement NCC1-382 were: (1) developing MSC/NASTRAN DMAP language scripts to implement the Soize fuzzy structures approach for modeling the dynamics of complex structures; (2) benchmarking the results of the new code to those for a cantilevered beam in the literature; and (3) testing and validating the new code by comparing the fuzzy structures results to NASA Langley experimental and conventional finite element results for two model test structures representative of aircraft fuselage sidewall construction: (A) a small aluminum test panel (SLP, single longeron panel) with a single longitudinal stringer attached with bolts; and (B) a 47 by 72 inch flat aluminum fuselage panel (AFP, aluminum fuselage panel) including six longitudinal stringers and four frame stiffeners attached with rivets.

Sparrow, Victor W.↗

DOE OSTI2021

Combination of deep neural network with attention mechanism enhances the explainability of protein contact prediction

Abstract Deep learning has emerged as a revolutionary technology for protein residue‐residue contact prediction since the 2012 CASP10 competition. Considerable advancements in the predictive power of the deep learning‐based contact predictions have been achieved since then. However, little effort has been put into interpreting the black‐box deep learning methods. Algorithms that can interpret the relationship between predicted contact maps and the internal mechanism of the deep learning architectures are needed to explore the essential components of contact inference and improve their explainability. In this study, we present an attention‐based convolutional neural network for protein contact prediction, which consists of two attention mechanism‐based modules: sequence attention and regional attention. Our benchmark results on the CASP13 free‐modeling targets demonstrate that the two attention modules added on top of existing typical deep learning models exhibit a complementary effect that contributes to prediction improvements. More importantly, the inclusion of the attention mechanism provides interpretable patterns that contain useful insights into the key fold‐determining residues in proteins. We expect the attention‐based model can provide a reliable and practically interpretable technique that helps break the current bottlenecks in explaining deep neural networks for contact prediction. The source code of our method is available at https://github.com/jianlin-cheng/InterpretContactMap .

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

Characterizing and improving the performance of molten-salt-steam heat exchangers in concentrating solar power plants

Shell-and-tube heat exchangers (HXs) for steam generation from molten salts in concentrating solar power (CSP) plants experience thermal fatigue due to significant temperature gradients and inherent transient operation. Molten salt-steam HX design lifespans exceed actual lifespans, and, as a consequence, designers overpredict plant profitability and operators neglect appropriate prescriptions to optimize these lifetimes. Here, this study refines HX lifespan estimates with data benchmarked against thermal-fluid mechanical modeling of stress and accumulated fatigue. Reduced-order thermal models of the molten salt-steam, shell-and-tube evaporator and superheater predict transient temperature profiles along the two HXs salt-steam flow paths. The modeled evaporator and superheater temperature profiles enable assessment of cyclic stresses within the HX tubesheets, where molten-salt HX failures are most common. Evaporator and superheater performance data from a current 110 MW elec commercial CSP plant provide a basis for validating the reduced-order HX models. HX life predictions derived from stochastic failure distributions serve as inputs for simulating and optimizing existing plant operations. The impact of the updated lifespans on overall plant revenue depends on operating scenarios. This study suggests that typical ramping rates for a CSP plant with a high-temperature Rankine cycle result in an evaporator and superheater life of approximately 10 and 25 years, respectively, compared to the design target of 30 years. Reduced HX lifespans decrease operational plant revenue on average by 4.6-5.1%. Furthermore, there may be as many as four HX replacements over the 30-year lifetime of the plant; and, purchase agreement loss due to failure to meet contractual production requirements can have ramifications that include the risk of bankruptcy.

14 SOLAR ENERGY↗

DOE OSTI2020

On the Network Topology of Cross-Linked Acrylate Photopolymers: A Molecular Dynamics Case Study

A reactive molecular dynamics approach is used to simulate cross-linking of acrylate polymer networks. By employing the same force field and reactive scheme and studying three representative multifunctional acrylate monomers, we isolate the importance of the nonreactive moieties within these model monomers. Analyses of reactive trajectories benchmark the estimated gel points, cyclomatic character, and spatially resolved cross-linking tendencies of the acrylates as a function of conversion. The insights into the similarities and differences of the polymerization and resulting networks presented here suggest molecular mechanics as a useful tool in the rational design of photopolymerization resins.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2021

Reduced Complexity Model Intercomparison Project Phase 2: Synthesizing Earth System Knowledge for Probabilistic Climate Projections

Abstract Over the last decades, climate science has evolved rapidly across multiple expert domains. Our best tools to capture state‐of‐the‐art knowledge in an internally self‐consistent modeling framework are the increasingly complex fully coupled Earth System Models (ESMs). However, computational limitations and the structural rigidity of ESMs mean that the full range of uncertainties across multiple domains are difficult to capture with ESMs alone. The tools of choice are instead more computationally efficient reduced complexity models (RCMs), which are structurally flexible and can span the response dynamics across a range of domain‐specific models and ESM experiments. Here we present Phase 2 of the Reduced Complexity Model Intercomparison Project (RCMIP Phase 2), the first comprehensive intercomparison of RCMs that are probabilistically calibrated with key benchmark ranges from specialized research communities. Unsurprisingly, but crucially, we find that models which have been constrained to reflect the key benchmarks better reflect the key benchmarks. Under the low‐emissions SSP1‐1.9 scenario, across the RCMs, median peak warming projections range from 1.3 to 1.7°C (relative to 1850–1900, using an observationally based historical warming estimate of 0.8°C between 1850–1900 and 1995–2014). Further developing methodologies to constrain these projection uncertainties seems paramount given the international community's goal to contain warming to below 1.5°C above preindustrial in the long‐term. Our findings suggest that users of RCMs should carefully evaluate their RCM, specifically its skill against key benchmarks and consider the need to include projections benchmarks either from ESM results or other assessments to reduce divergence in future projections.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

SmileyLlama: modifying large language models for directed chemical space exploration

Here we show that large language models (LLMs) can be transformed via supervised fine-tuning of engineered prompts into SmileyLlama for exploring the chemical space of drug molecules. We benchmark SmileyLlama against pretrained LLMs and chemical language models trained from scratch for generating valid and novel drug-like molecules, and use direct preference optimization to both improve SmileyLlama’s adherence to a prompt and as part of the iMiner reinforcement learning framework to predict molecules with optimized three-dimensional conformations and high binding affinity to drug targets. By training an LLM to speak directly as a chemical language model, while retaining most of its natural language capabilities, we show that SmileyLlama can reliably generate molecules with user-specified properties rather than acting only as a chatbot with knowledge of chemistry or as a virtual assistant. While SmileyLlama is geared toward drug discovery, the supervised fine-tuning/direct preference optimization/LLM framework can be extended to other chemical, biological and materials applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Radiation and heat transport in divergent shock–bubble interactions

Shock–bubble interactions (SBIs) are important across a wide range of physical systems. In inertial confinement fusion, interactions between laser-driven shocks and micro-voids in both ablators and foam targets generate instabilities that are a major obstacle in achieving ignition. Experiments imaging the collapse of such voids at high energy densities (HED) are constrained by spatial and temporal resolution, making simulations a vital tool in understanding these systems. In this study, we benchmark several radiation and thermal transport models in the xRAGE hydrodynamic code against experimental images of a collapsing mesoscale void during the passage of a 300 GPa shock. We also quantitatively examine the role of transport physics in the evolution of the SBI. This allows us to understand the dynamics of the interaction at timescales shorter than experimental imaging framerates. We find that all radiation models examined reproduce empirical shock velocities within experimental error. Radiation transport is found to reduce shock pressures by providing an additional energy pathway in the ablation region, but this effect is small (∼1% of total shock pressure). Employing a flux-limited Spitzer model for heat conduction, we find that flux limiters between 0.03 and 0.10 produce agreement with experimental velocities, suggesting that the system is well-within the Spitzer regime. Higher heat conduction is found to lower temperatures in the ablated plasma and to prevent secondary shocks at the ablation front, resulting in weaker primary shocks. Finally, we confirm that the SBI-driven instabilities observed in the HED regime are baroclinically driven, as in the low energy case.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗