Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

Towards Realistic Models of Heterogeneous Catalysis: Simulations of Oxidation Catalysis from First Principles

Among the most significant developments in heterogeneous catalysis in the last 20 years is the emergence of microkinetic models, often parameterized from density functional theory (DFT) calculations, used to quantify observed catalyst performance and to guide the discovery of new catalytic materials. While DFT directly reports binding energies and elementary step activation energies, the free energies that enter into microkinetic models must be computed from additional approximations. Frequently these free energy approximations assume ideal behavior—that adsorbates do not interact with one another, or that adsorbates are immobile or vibrate harmonically about a binding site. These assumptions can and do have an impact on predicted catalyst performance and potentially even on predicted trends. Our work relates to three categories of non-idealities, explored in the context of nitrogen oxidation and reduction catalysis on metal surfaces. One component relates to adsorbate translational free energy, the contribution least well described by conventional models, in which we develop modeling approaches that improve accuracy with limited increase in computational expense. A second component relates adsorbate-adsorbate interactions, in which we develop benchmark on-lattice interaction models and compare kinetic predictions with conventional mean-field models, in an effort to develop more robust coverage-dependent mean field modeling approaches. The last relates the most fundamental assumption of all—that of energy equipartition—in an effort to rationalize and guide plasma-enhanced catalysis.

99 GENERAL AND MISCELLANEOUS↗

Development of a sensor coordinated kinematic model for neural network controller training

A robotic benchmark problem useful for evaluating alternative neural network controllers is presented. Specifically, it derives two camera models and the kinematic equations of a multiple degree of freedom manipulator whose end effector is under observation. The mapping developed include forward and inverse translations from binocular images to 3-D target position and the inverse kinematics of mapping point positions into manipulator commands in joint space. Implementation is detailed for a three degree of freedom manipulator with one revolute joint at the base and two prismatic joints on the arms. The example is restricted to operate within a unit cube with arm links of 0.6 and 0.4 units respectively. The development is presented in the context of more complex simulations and a logical path for extension of the benchmark to higher degree of freedom manipulators is presented.

Jorgensen, Charles C.↗

Verified, Archived Library of Inputs and Data (VALID) Online Repository [Slides]

VALID continues to serve as a library of high-quality models used for evaluating SCALE and nuclear data. Large number of cases are in the pipeline. Online repository is available for use. Future plans include simplifying the process for adding cases, including additional models outside the ICSBEP Handbook (HTC models, DNCSH models, International Handbook of Evaluated Reactor Physics Benchmark Experiments [IRPhE], SINBAD, etc.), and increasing the availability of models and results for external users.

KENO↗

Reactor physics benchmark experiments at the JSI TRIGA MARK II reactor - Current status and future outlook

Full text of publication follows. With the development of new high-fidelity computational methods, improvement of nuclear data, and multiphysics modelling, there is an increased need for benchmark experiments to experimentally validate the models, methods and input data. Many of the nuclear facilities designed to perform reactor physics benchmark experiments have been shut down. Therefore, research reactors offer a great opportunity for benchmark experiments, if they are well designed and performed with great care and accuracy. In this presentation we provide an overview of the past and ongoing activities related to benchmark experiments at the Jozef Stefan Institute TRIGA Mark II research reactor. The following experiments have been performed: criticality with fresh fuel, {sup 197}Au(n,γ) and {sup 27}Al(n,α) reaction rates in irradiation channels, absolute and relative {sup 197}Au(n,γ), {sup 235}U(n,f) and {sup 238}U(n,f) reaction rates in the core, burnup, kinetic parameters, control rod worth, isothermal reactivity coefficient, self-shielding, slow and fast (pulse) transients, nuclear heating, delayed and prompt gamma ray production, temperature profiles for multi-physics. Since the existing fleet of research reactors is ageing very rapidly and new experiments are needed, new research reactors should be designed and built to meet the needs of future advanced reactors, education and training, and other technologies in the coming years. We will review planned activities at the JSI TRIGA reactors and plans for the new research reactor in Slovenia. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Test Cases for Modeling and Validation of Structures with Piezoelectric Actuators

A set of benchmark test articles were developed to validate techniques for modeling structures containing piezoelectric actuators using commercially available finite element analysis packages. The paper presents the development, modeling, and testing of two structures: an aluminum plate with surface mounted patch actuators and a composite box beam with surface mounted actuators. Three approaches for modeling structures containing piezoelectric actuators using the commercially available packages: MSC/NASTRAN and ANSYS are presented. The approaches, applications, and limitations are discussed. Data for both test articles are compared in terms of frequency response functions from deflection and strain data to input voltage to the actuator. Frequency response function results using the three different analysis approaches provided comparable test/analysis results. It is shown that global versus local behavior of the analytical model and test article must be considered when comparing different approaches. Also, improper bonding of actuators greatly reduces the electrical to mechanical effectiveness of the actuators producing anti-resonance errors.

Reaves, Mercedes C.↗

A High-Fidelity Electromagnetic Transient Model of Inverter-based Resources Integrated to an IEEE-9 Bus System for Benchmarking Studies

An inverter-based resource (IBR) is a source of electricity that is asynchronously connected to the electrical grid via an electronic power converter. These power sources lack the intrinsic behaviors of the standard power plants, presenting specific challenges to system stability. These power sources lack the intrinsic behaviors of the standards power plants and their features are almost entirely defined by the control strategy used on them, presenting specific challenges to system stability as their penetration increases into the already fragile Bulk Power System (BPS). Utilizing renewable energy has a great upside but the electrical industry needs to understand the requirements like performing electromagnetic transient (EMT) simulation studies in planning and/or in interconnection studies. This paper presents the development of a high-fidelity EMT model based on commercial equipment and field parameters, which at the same time aims to study the increasing penetration of IBRs and their impact on the actual BPS. The open-source EMT model developed will be a benchmarking example that can be used to evaluate algorithms in the power grid (including evaluating grid-forming controls, power system monitoring and operation, etc.).

Martinez Montejano, Misael↗

The accuracy of multi-group models for nonlocal electron transport in magnetized plasmas

In the extreme conditions of inertial confinement fusion experiments, heat flow plays a vital role, but local diffusive models frequently break down and overestimate the heat flow. The situation becomes more complicated again in the significant magnetic fields generated during laser–plasma interactions or in magnetized fusion schemes. Accurate non-local and magnetized heat flow computations can be carried out using Vlasov–Fokker–Planck (VFP) simulations, but these are computationally expensive. There is, therefore, significant interest in using faster multi-group models to accurately calculate the non-local heat flow in magnetized plasmas. We benchmark two such multi-group models for calculating the heat flow, M1 and hybrid-AWBS-BGK, against diffusive models and full VFP simulations, before applying the models to realistic example test cases, both magnetized and unmagnetized. We find that the multi-group models generally perform very well for moderate non-localities up to kλmfp∼0.01, but the computational cost increases dramatically. hybrid-AWBS-BGK performs more effectively than M1 at high non-localities, up to kλmfp∼1, due to its adaptive solver and robust P1 closure, but tends to fail in very strong magnetic fields. Both codes are much faster than VFP simulations but are still slow in steep temperature gradients.

Arran, C. (ORCID:0000000286448118)↗

CoderData

Benchmark dataset that harmonizes drug response data across thousands of patient samples and cancer model systems enabling the training and benchmarking of AI/ML algorithms at scale.

Gosline, Sara↗

Evaluation of Oak Ridge National Laboratory Health Physics Research Reactor Operation Data for Critical Benchmark Creation [Slides]

During the evaluation of Oak Ridge National Laboratory Health Physics Research Reactor operation data for critical benchmark creation, a detailed, functional SCALE model was built, and experimental data was evaluated for the creation of a critical benchmark. The estimated experimental uncertainty is about 3800 pcm, which is considered very high. The relative difference between expected and calculated keff values is about 1.5 %, also considered very high. It was concluded that a good quality critical benchmark worthy of the ICSBEP standards cannot be created from HPRR data in the present conditions. Locating the HPRR fuel to obtain an uncertainty on the density would aid in solving this issue.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Experiments for SINBAD: Evaluation of Oak Ridge Health Physics Research Reactor Operation Data for CAAS Benchmark Creation [Slides]

This report was a real information preservation and dissemination work with a lot of legacy content that was found and used. There was an abundance of uncertainty, discrepancy, and contradictory information. Yet, a detailed, functional SCALE model was built, and the benchmark created is useful for shielding and CAAs validation work. Sulfur fluence C/E ratios are large (2 to 5), so different benchmark metrics were studied. Neutron fluence, element 57 dose, and other dosimetry responses at 3 meters C/E ratios are below 1.5 for bare and steel configurations. Additional promising metrics as dose per unit fluence and steel shield attenuation were computed.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

NEAMS Model Contributions in 2023 to the National Reactor Innovation Center Virtual Test Bed for Use by Industry and Other Stakeholders

The U.S. Department of Energy (DOE) Office of Nuclear Energy’s Advanced Modeling and Simulation (NEAMS) Program develops models of advanced reactor phenomena to demonstrate code applicability to challenging physics problems, drive code development through user assessment, and perform code verification and validation. Meanwhile, the U.S. DOE’s National Reactor Innovation Center (NRIC) hosts an open-source website and associated GitHub repository called the Virtual Test Bed (VTB) on which computational models for advanced reactors are documented and shared with the reactor community. This work documents NEAMS efforts to support industry adoption of advanced modeling tools through contribution of 10 NEAMS models to the NRIC Virtual Test Bed including models for the High Temperature Test Facility (HTTF), TRISO fuel failure in a microreactor, and multiphysics models of a molten chloride fast reactor, among others. The open sharing of these models benefits the reactor community by providing “best practice” examples using NEAMS tools for advanced reactor physics problems. In particular, the HTTF model is being used for code validation and benchmarking activities. The microreactor and molten chloride fast reactor models are representative of analysis that may be useful for current candidates of DOME and LOTUS, NRIC’s physical testbeds. This report summarizes and provides links to these new models, among others.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or aerial wildfire operations reports to better understand the risks present. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement. These applications can benefit from the use of state-of-the-art natural language processing techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of adaptation of NLP tools to the aviation domain by fine-tuning transformer based models using aviation data sets. In 2018, a novel language model based on neural units (also called transformers) was created and became known as “Bidirectional Encoder Representations from Transformers” or BERT. This architecture combined with large amounts of English training data and innovative semi-supervised training tasks set the standard for what would later emerge as Large Language Models. The performance of these models was further improved by hyperparameter tuning and refinement of the semi-supervised training task and resulted in “Robustly Optimized BERT Pre-training Approach through hyperparameter tuning” or RoBERTa models. These pre-trained Large Language Models proved to be useful for a wide variety of natural language processing tasks such as text classification and question answering through a process called fine-tuning. The transformer architecture with pre-trained weights served as the basis with the last few layers replaced with layers fine-tuned to perform a new task e.g., a layer that provides a label for the entire input text. This process of fine-tuning can also be used to adapt the models to new domains; e.g., BioBERT started with the pre-trained BERT model and was completed by additional fine-tuning and training on biomedical documents. Transformer-based architectures can also be used to create rich representations of text called embeddings which can serve as the input to other machine learning models. This allows simpler algorithms such as logistic regression to use context-rich representations of the text while still remaining quick to train and evaluate. In the world of aviation, there is a growing demand for natural language processing and understanding but the domain presents unique challenges. Due to the technical content (and specialized language) of most aviation documents, fine-tuning pre-trained Large Language Models to specific tasks has not met the benchmark on natural language processing tasks set by simpler models trained from scratch on the data. To address this deficiency, this paper evaluates the improvements from fine-tuning a Large Language Model on a large set of aviation documents using the original semi-supervised training tasks before performing specific natural language tasks. In fine-tuning, a domain-specific dataset is used on the original training task but with the pre-trained Large Language Model instead of starting from a random initialization. This approach allows the model to be adapted to the specific domain language without discarding the information gained from training on general English data. This paper utilized two major dataset types to train and assess the RoBERTa fine-tuning performance. The first are 7,057 Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the semi-supervised fine-tuning. The second type is the 494 document classification labels to be used for evaluation. This down-stream evaluation aims to show the performance of the fine-tuned model, better understand how much data is needed for an effective fine-tuning, and how fine-tuning can be adapted for different applications in-the domain. After semi-supervised training, evaluation begins by encoding the documents for classification using the fine-tuned RoBERTa model. Then a logistic regression classifier is trained to label the document type and compared against our ground truth labels. This currently leads to a 82.8% accuracy on 10-fold cross validation showing improvement over baseline RoBERTa which achieved 81.0%. We plan to measure the improvements on additional tasks and it is expected that these improvements will lead to more robust models that can tackle the natural language processing challenges present in aviation datasets.

ATM↗

Modeling betatron radiation using particle-in-cell codes for plasma wakefield accelerator diagnostics

The analysis of plasma wakefield acceleration experimental measurements, particularly in the characterization of photons emitted through the betatron radiation mechanism, requires the development of accurate numerical models. These computational models are crucial for supporting modern instrumentation designed to measure the single-shot, double-differential angular-energy radiation spectra resulting from interactions between beams and plasmas. Motivated by the needs of such applications, this paper presents detailed numerical models of betatron radiation generated in beam-plasma acceleration experiments. These models are based on the integration of the Liénard-Wiechert (LW) potentials, applied to computed particle trajectories. The particle trajectories are generated using three distinct methods: first, by tracking particles through idealized fields in the blowout regime of PWFA; second, by obtaining trajectories using the fast quasistatic particle-in-cell (PIC) code quickpic; and third, obtaining trajectories from the fully self-consistent PIC code osiris. To ensure the accuracy and reliability of these models, the paper includes various benchmark tests using analytical expressions, as well as employing the PIC code epoch, which takes an alternative approach by using a Monte Carlo quantum electrodynamics (QED)-based radiation model. Additionally, the paper presents simulations of the expected experimental betatron radiation spectra, taking into account parameters relevant to PWFA and plasma photocathode experiments at the SLAC FACET-II facility.

Yadav, M. [University of California, Los Angeles, ↗

FORCE Development Status Update: Vertical Integration and Benchmarking of System Dynamics

Recent efforts to establish effective models for grid energy analysis, especially given the increase in variable renewable energy (VRE) sources and the economic challenges faced by traditional nuclear energy, have generated new technological considerations. One effort to improve the economic viability of nuclear power involves investigating integrated energy systems (IES) which include secondary energy systems that introduce flexibility and secondary market possibilities to existing and perceived future nuclear energy generation technologies. To analyze the technical and economic potential of IES, the Framework for Optimization of Resources and Economics (FORCE) tool suite was developed through a collaboration among national laboratories. Within the FORCE tool suite, the Holistic Energy Resource Optimization Network (HERON) provides algorithms for analyzing the long-term viability of potential IES technologies, while HYBRID provides algorithms and models to achieve high-resolution analysis of coupling physics over a short time period. Continued maturing of the FORCE tool suite requires further interconnections between the various tools in the suite in order to ensure consistent analysis. Analyses performed by applying HYBRID to transient process modeling should be easily harvestable as inputs to HERON analyses. The first item in this status update is a demonstration of an automated data pipeline for loading data from HYBRID into HERON analyses. Application of HYBRID results to HERON, as part of using the FORCE tool suite, relies on robust modeling assumptions for the various models included in HYBRID. The second result of this status update is the benchmarking and validation of cost and operational data, with a particular focus on natural gas energy generators. These generators are benchmarked with a focus on contrasting them with proposed thermal energy storage (TES) technologies.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

AFRL Additive Manufacturing Modeling Series: Challenge 1, Characterization of Residual Strain Distribution in Additively-Manufactured Metal Parts Using Energy-Dispersive Diffraction

A machine component fabricated by additive manufacturing (AM) of metal powders often possesses steep residual stress gradients with significant magnitudes due to large temperature gradients that transpire within a localized area during fabrication. These processing-induced residual stresses can cause distortion of the part, and if they are of significant magnitude, they could induce cracking of the component, degrade printability, and/or diminish subsequent mechanical performance. The ability to predict these residual stresses imparted by AM is an important step in permeating AM technology for advanced manufacturing. Calibration and validation of AM process models used for prediction are, therefore, a critical step in understanding the origin and mitigating the challenges associated with residual stresses inherent to the AM process. In the present work, the residual strain distributions in components with simple geometries fabricated by a laser powder bed fusion (LBPF) process were characterized non-destructively using energy-dispersive X-ray diffraction in support of the US Air Force Research Laboratory Additive Manufacturing Modeling Challenge Series Groeber et al. (JOM 70:441-448, 2018). The measurement setup and approach are described in detail so that the data can be used as a benchmark to calibrate and validate models for the prediction of macro-scale residual stresses due to the LPBF process.

36 MATERIALS SCIENCE↗

PyApprox: A software package for sensitivity analysis, Bayesian inference, optimal experimental design, and multi-fidelity uncertainty quantification and surrogate modeling

PyApprox is a Python-based one-stop-shop for probabilistic analysis of numerical models such as those used in the earth, environmental and engineering sciences. Easy to use and extendable tools are provided for constructing surrogates, sensitivity analysis, Bayesian inference, experimental design, and forward uncertainty quantification. The algorithms implemented represent a wide range of methods for model analysis developed over the past two decades, including recent advances in multi-fidelity approaches that use multiple model discretizations and/or simplified physics to significantly reduce the computational cost of various types of analyses. An extensive set of Benchmarks from the literature is also provided to facilitate the easy comparison of new or existing algorithms for a wide range of model analyses. Here, this paper introduces PyApprox and its various features, and presents results demonstrating the utility of PyApprox on a benchmark problem modeling the advection of a tracer in groundwater.

54 ENVIRONMENTAL SCIENCES↗

Validation and Verification of Operational Land Analysis Activities at the Air Force Weather Agency

The NASA developed Land Information System (LIS) is the Air Force Weather Agency's (AFWA) operational Land Data Assimilation System (LDAS) combining real time precipitation observations and analyses, global forecast model data, vegetation, terrain, and soil parameters with the community Noah land surface model, along with other hydrology module options, to generate profile analyses of global soil moisture, soil temperature, and other important land surface characteristics. (1) A range of satellite data products and surface observations used to generate the land analysis products (2) Global, 1/4 deg spatial resolution (3) Model analysis generated at 3 hours. AFWA recognizes the importance of operational benchmarking and uncertainty characterization for land surface modeling and is developing standard methods, software, and metrics to verify and/or validate LIS output products. To facilitate this and other needs for land analysis activities at AFWA, the Model Evaluation Toolkit (MET) -- a joint product of the National Center for Atmospheric Research Developmental Testbed Center (NCAR DTC), AFWA, and the user community -- and the Land surface Verification Toolkit (LVT), developed at the Goddard Space Flight Center (GSFC), have been adapted to operational benchmarking needs of AFWA's land characterization activities.

Shaw, Michael↗

Improving Reanalyses Using TRMM and SSM/I-Derived Precipitation and Total Precipitable Water Observations

Global reanalyses currently contain significant errors in the primary fields of the hydrological cycle such as precipitation, evaporation, moisture, and the related cloud fields, especially in the tropics. The Data Assimilation Office (DAO) at the NASA Goddard Space Flight Center has been exploring the use of rainfall and total precipitable water (TPW) observations from the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) and the Special Sensor Microwave/ Imager (SSM/I) instruments to improve these fields in reanalyses. The DAO has developed a "1+1"D procedure to assimilate 6-hr averaged rainfall and TPW into the Goddard Earth Observing System (GEOS) Data Assimilation System (DAS). The algorithm is based on a 6-hr time integration of a column version of the GEOS DAS. The "1+1" designation refers to one spatial dimension plus one temporal dimension. The scheme minimizes the least-square differences between the satellite-retrieved rain rates and those produced by the column model over the 6-hr analysis window. The control variables are analysis increments of moisture within the Incremental Analysis Update (IAU) framework of the GEOS DAS. This 1+1D scheme, in its generalization to four dimensions, is related to the standard 4D variational assimilation but differs in its choice of the control variable. Instead of estimating the initial condition at the beginning of the assimilation cycle, it estimates the constant IAU forcing applied over a 6-hr assimilation cycle. In doing so, it imposes the forecast model as a weak constraint in a manner similar to the variational continuous assimilation techniques. We present results from an experiment in which the observed rain rate and TPW are assumed to be "perfect". They show that assimilating the TMI and SSM/I-derived surface precipitation and TPW observations improves not only the precipitation and moisture fields but also key climate parameters directly linked to convective activities such as clouds, the outgoing longwave radiation, and the large-scale circulation in the tropics. In particular, assimilating these data types reduce the state-dependent systematic errors in the assimilated products. The improved analysis also leads to a better short-range forecast, but the impact is modest compared with improvements in the time-averaged fields. These results suggest that, in the presence of biases and other errors of the forecast model, it is possible to improve the time-averaged "climate content" in the assimilated data without comparable improvements in the short-range forecast skill. Results of this experiment provide a useful benchmark for evaluating error covariance models for optimal use of these data types.

Hou, Arthur Y.↗