Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗

Benchmark and Framework for Encouraging Research on Multi-Threaded Testing Tools

A problem that has been getting prominence in testing is that of looking for intermittent bugs. Multi-threaded code is becoming very common, mostly on the server side. As there is no silver bullet solution, research focuses on a variety of partial solutions. In this paper (invited by PADTAD 2003) we outline a proposed project to facilitate research. The project goals are as follows. The first goal is to create a benchmark that can be used to evaluate different solutions. The benchmark, apart from containing programs with documented bugs, will include other artifacts, such as traces, that are useful for evaluating some of the technologies. The second goal is to create a set of tools with open API s that can be used to check ideas without building a large system. For example an instrumentor will be available, that could be used to test temporal noise making heuristics. The third goal is to create a focus for the research in this area around which a community of people who try to solve similar problems with different techniques, could congregate.

Havelund, Klaus↗

Investigation of thermal hydraulic behavior of the High Temperature Test Facility's lower plenum via large eddy simulation

A high-fidelity computational fluid dynamics (CFD) analysis was performed using the Large Eddy Simulation (LES) model for the lower plenum of the High–Temperature Test Facility (HTTF), a ¼ scale test facility of the modular high temperature gas-cooled reactor (MHTGR) managed by Oregon State University. In most next–generation nuclear reactors, thermal stress due to thermal striping is one of the risks to be curiously considered. This is also true for HTGRs, especially since the exhaust helium gas temperature is high. In order to evaluate these risks and performance, organizations in the United States led by the OECD NEA are conducting a thermal hydraulic code benchmark for HTGR, and the test facility used for this benchmark is HTTF. HTTF can perform experiments in both normal and accident situations and provide high-quality experimental data. However, it is difficult to provide sufficient data for benchmarking through experiments, and there is a problem with the reliability of CFD analysis results based on Reynolds–averaged Navier–Stokes to analyze thermal hydraulic behavior without verification. To solve this problem, high-fidelity 3-D CFD analysis was performed using the LES model for HTTF. It was also verified that the LES model can properly simulate this jet mixing phenomenon via a unit cell test that provides experimental information. As a result of CFD analysis, the lower the dependency of the sub-grid scale model, the closer to the actual analysis result. In the case of unit cell test CFD analysis and HTTF CFD analysis, the volume-averaged sub-grid scale model dependency was calculated to be 13.0% and 9.16%, respectively. As a result of HTTF analysis, quantitative data of the fluid inside the HTTF lower plenum was provided in this paper. As a result of qualitative analysis, the temperature was highest at the center of the lower plenum, while the temperature fluctuation was highest near the edge of the lower plenum wall. The power spectral density of temperature was analyzed via fast Fourier transform (FFT) for specific points on the center and side of the lower plenum. FFT results did not reveal specific frequency-dominant temperature fluctuations in the center part. It was confirmed that the temperature power spectral density (PSD) at the top increased from the center to the wake. The vortex was visualized using the well-known scalar Q-criterion, and as a result, the closer to the outlet duct, the greater the influence of the mainstream, so that the inflow jet vortex was dissipated and mixed at the top of the lower plenum. Additionally, FFT analysis was performed on the support structure near the corner of the lower plenum with large temperature fluctuations, and as a result, it was confirmed that the temperature fluctuation of the flow did not have a significant effect near the corner wall. In addition, the vortices generated from the lower plenum to the outlet duct were identified in this paper. It is considered that the quantitative and qualitative results presented in this paper will serve as reference data for the benchmark.

97 MATHEMATICS AND COMPUTING↗

Metal hydride composition-derived parameters as machine learning features for material design and H 2 storage

Though hydrogen is a promising energy carrier for a green future, many challenges persist. One is the difficulty in engineering storage solutions, with metal hydrides being a leading contender among solid-state strategies. To facilitate efficient searching of candidate materials, ridge regression, simple decision trees, random forest ensembles, and gradient boosting ensembles were employed to predict the energy of formation, with the random forest ensemble resulting in the lowest test set error. First, two public databases, Materials Project and HydPark, were searched for metal hydrides. Feature engineering was performed before the models were developed, resulting in electronegativity, density, atomic density, d-character, f-character, band gap, hydrogen weight fraction, magnetization, temperature, and pressure being retained. The models were then benchmarked by the lowest test error before a random forest ensemble was used to populate entries missing energy of formation. Furthermore, all were then scored by hydrogen storage capacity and energy of formation suitability. Readily available features including several derived from only the chemical formula which were found to be highly predictive. and so are promising for high-throughput screening of arbitrary novel hydride formulations and blends for thermodynamic feasibility.

25 ENERGY STORAGE↗

Third and Final Shuttle Mission of the Isothermal Dendritic Growth Experiment Conducted: Highest Supercooling Ever Recorded Achieved

Dendrites describe the treelike crystal morphology commonly assumed in metals and alloys that freeze from supercooled or supersaturated melts. There remains a high level of engineering interest in dendritic solidification because the size, shape, and orientation of the dendrites determine the final microstructure of a material. It is the microstructure that then determines the physical properties of cast or welded products. Although it is well known that dendritic growth is controlled by the transport of latent heat from the moving solid-liquid interface, an accurate and predictive model has not yet been developed. The effects of gravity-induced convection on the transfer of heat from the interface have prevented adequate testing, under terrestrial conditions, of solidification models. The Isothermal Dendritic Growth Experiment (IDGE) constituted a series of three microgravity experiments flown aboard the Space Shuttle Columbia. The apparatus was used to grow and record dendrite solidification in the absence of gravity-induced convective heat transfer, thereby producing a wealth of benchmark-quality data for testing solidification models and theories.

Glicksman, Martin E.↗

Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List

As a rule, Top 500 class supercomputers are extensively benchmarked as part of their acceptance testing process. However, barring publicly posted LINPACK / HPCG results, most benchmark results are often inaccessible outside the hosting institution. Moreover, these higher level benchmarks do not provide easy answers to common questions such as “What is the realizable memory bandwidth?” or “What is the launch latency on the accelerator?” To partially address these issues, we executed selected single-node micro-benchmarks — focused on latencies and memory bandwidth — on every US Department of Energy system above rank 150 of the June 2023 Top 500 list, excepting NERSC’s Cori and ORNL’s Frontier TDS (now decommissioned or repurposed). We hope to provide an easy “first stop” reference for users of current Top 500 systems and inspire users and administrators of other Top 500 systems to similarly compile and make available benchmark results for their systems.

Siefert, Christopher M.↗

Space network scheduling benchmark: A proof-of-concept process for technology transfer

This paper describes a detailed proof-of-concept activity to evaluate flexible scheduling technology as implemented in the Request Oriented Scheduling Engine (ROSE) and applied to Space Network (SN) scheduling. The criteria developed for an operational evaluation of a reusable scheduling system is addressed including a methodology to prove that the proposed system performs at least as well as the current system in function and performance. The improvement of the new technology must be demonstrated and evaluated against the cost of making changes. Finally, there is a need to show significant improvement in SN operational procedures. Successful completion of a proof-of-concept would eventually lead to an operational concept and implementation transition plan, which is outside the scope of this paper. However, a high-fidelity benchmark using actual SN scheduling requests has been designed to test the ROSE scheduling tool. The benchmark evaluation methodology, scheduling data, and preliminary results are described.

Moe, Karen↗

The Isothermal Dendritic Growth Experiment (IDGE): USMP-4 One-Year-Report

Dendrites describe the tree-like crystal morphology commonly assumed in many material systems--particularly in metals and alloys that freeze from supercooled or supersaturated melts. There remains a high level of engineering interest in dendritic solidification because of the role of dendrites in the determination of cast alloy microstructures. Microstructure plays a key role in determining the physical properties of cast or welded products. In addition, dendritic solidification provides an example of non-equilibrium physics and is one of the simplest non-trivial examples of dynamic pattern formation, where an amorphous melt, under simple starting conditions, evolves into a complex ramified microstructure. Although it is well-known that dendritic growth is controlled by the transport of latent heat from the moving solid-melt interface as the dendrite advances into a supercooled melt, an accurate, and predictive model has not been developed. Current theories consider: 1) the transfer of heat or solute from the solid-liquid interface into the melt, and 2) the interfacial crystal growth and growth selection physics for the interface. However, the effects of gravity-induced convection on the transfer of heat from the interface prevent either element from being adequately tested solely under terrestrial conditions. The Isothermal Dendritic Growth Experiment (IDGE) constituted a series of three NASA-supported microgravity experiments, all of which flew aboard the space shuttle, Columbia. This experimental space flight series was designed and operated to grow and record dendrite solidification in the absence of gravity-induced convective heat transfer, and thereby produce a wealth of benchmark-quality data for testing solidification scaling laws. The data collection from the on-orbit phase of the IDGE flight series is now complete. We are currently completing analyses and moving towards final data archiving.

Glicksman, M. E.↗

Translating an AI application from Lisp to Ada: A case study

A set of benchmarks was developed to test the performance of a newly designed computer executing both Lisp and Ada. Among these was AutoClassII -- a large Artificial Intelligence (AI) application written in Common Lisp. The extraction of a representative subset of this complex application was aided by a Lisp Code Analyzer (LCA). The LCA enabled rapid analysis of the code, putting it in a concise and functionally readable form. An equivalent benchmark was created in Ada through manual translation of the Lisp version. A comparison of the execution results of both programs across a variety of compiler-machine combinations indicate that line-by-line translation coupled with analysis of the initial code can produce relatively efficient and reusable target code.

Davis, Gloria J.↗

Benchmark Exercise Report for Experimental Study of Bubble Scrubbing in Water Coolant Pool

Mechanistic assessments of radionuclide release during postulated accidents are expected to be included in advanced reactor license applications. The mechanistic source term (MST) provides an opportunity for vendors to realistically evaluate the radiological consequences of an incident, and may aid in justifying reduced emergency planning zones and plant sites. However, the development of MSTs for advanced nuclear reactors is challenging because there are numerous phenomena that can affect the transport and retention of radionuclides. As part of a trial MST assessment for a metal-fueled, pool-type sodium cooled fast reactor (SFR), led by Argonne National Laboratory, a simplified radionuclide transport code (SRT code) was developed, which includes models to estimate the quantity of fission product aerosols scrubbed in the sodium pool during postulated accident scenarios. In a pool-type SFR, when fission products are released into the coolant pool due to failure of fuel pins, most of the radionuclides are scrubbed by the coolant pool, but some have the potential to migrate to the cover gas region through entrainment within gas bubbles. The SRT code contains a model that evaluates this scrubbing behavior and calculates the fraction of fission product aerosols that reach the cover gas. Due to a lack of available validation data for sodium pool scrubbing, the U.S. Department of Energy funded an experiment at the University of Wisconsin-Madison to measure aerosol scrubbing by injecting air bubbles containing aerosol into a coolant pool. Prior to performing an experiment with liquid sodium, a water loop experiment was performed. Their experiment evaluated the effect of changing the aerosol size, aerosol density, aerosol concentration, bubble size, and pool depth on the aerosol scrubbing efficiency of the pool. In this benchmark experiment, the base tests were conducted by repeated tests of isolated bubbles. Afterwards, more prototypic tests with bubble swarms were performed to evaluate the interactions between the bubbles. The bubble swarm test was able to confirm that a larger amount of aerosol scrubbing occurred than the single bubble test. It was also confirmed that as the bubble size, aerosol density, and pool height increase, the extent of pool scrubbing also increases and does not change with the aerosol concentration. In addition, since the degree of scrubbing is the lowest at aerosol sizes between 0.01 and 1 μm, that is, the largest amount of aerosol is emitted, it was confirmed that the analysis of this size in MST is the most important. This benchmark experiment informs the direction of future sodium experiments.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

VERA BWR progression problems

During the first phase of the Consortium for Advanced Simulation of Light Water Reactors (CASL) program, the Virtual Environment for Reactor Applications (VERA) was developed with a focus on capabilities for high-fidelity, multiphysics simulation of pressurized water reactors (PWRs). During this development effort, a set of progression problems was created ranging from smaller pin cell calculations to larger 3D full-core calculations. These progression problems helped to guide the development of the software and served as benchmarks against which to test VERA. Since 2019, efforts have been made to extend the capabilities of VERA to model boiling water reactors (BWRs). Because BWR simulations come with many unique challenges, a set of BWR progression problems was developed to aid in this new effort. The BWR progression problems range from 2D lattice calculations to 3D mini-core problems, and reference neutronic solutions were computed using continuous-energy Monte Carlo codes. MPACT, one of the neutronics code in VERA, was benchmarked using the BWR progression problems. The code is capable of computing solutions to all problems. The eigenvalues computed by MPACT agree well with the Monte Carlo reference solutions. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

The X-43A Flight Research Program: Lessons Learned on the Road to Mach 10

During an aerospace engineer's undergraduate studies, he or she will attend classes in aerodynamics, thermodynamics, structures, stability and control, dynamics, design, propulsion, and computer science, along with the related courses in mathematics, physics, statistics, and chemistry required to understand the material. Upon graduation, the new engineer will have acquired a basic knowledge of how to build an aerospace vehicle. What only comes through experience, however, is the understanding of the inevitable imperfect process through which an aerospace vehicle is built. This is the adventure of turning a basic concept into functional hardware. Engineers working on a project must often deal with ambiguous situations. They are routinely asked by management to provide risk assessments of a project, yet even after careful analysis uncertainties remain. The project must be accomplished within finite limits of time and money. The question an engineer answers is whether the solution to potential problem is worth the cost and schedule delay, or if the solution might actually be worse than the problem it is meant to solve. Review protocols are established to ensure that an unknown has not been overlooked. But these cannot protect against an unknown unknown. Examples of these situations can be found in the history of the X-43A Hyper-X (Hypersonic Experiment) program. In this NASA project, a supersonic combustion ramjet (scramjet) engine was flight tested on a subscale vehicle. The X-43A Hyper-X Research Vehicle (HXRV) was launched from a B-52B mothership, then boosted to the test speed by a modified Pegasus rocket first stage, called the Hyper-X Launch Vehicle (HXLV). Once at the proper speed and altitude, the X-43A separated from the booster, stabilized itself, and then the engine test began. Although wind-tunnel scramjet engine tests had begun in the late 1950s, before the Hyper-X program there had never been an actual in-flight test of such an engine integrated with an appropriate airframe. Thus, while the scramjet had successfully operated in the artificial airflow of wind tunnels, the concept had yet to be proven in real air. These conditions meant changes in density and temperature, as well as changes in angle of attack and sideslip of a free-flying vehicle. A wind tunnel is limited in its ability to simulate these subtle factures, which have a major impact on almost any vehicle, but especially that of a scramjet's performance. The Hyper-X project was to provide a real-world benchmark of the ground test data. The full scale X-43A engine would be operated in the wind tunnel, and then flown, and the data from its operation would then be compared with projections. If these matched, the wind tunnel data would be considered a reliable design tool for future scramjet. If there were significant differences, the reasons for these would have to be identified. Until such information was available, scramjets would lack the technological maturity to be considered for future space launch or high-speed atmospheric flight vehicles.

Peebles, Curtis↗

INL Contributions to Draft HTTF Benchmark Specifications

The High-Temperature Test Facility (HTTF) is an integral effects thermal hydraulics test facility at Oregon State University designed as a 1/4 length scale model of the Modular High-Temperature Gas-Cooled Reactor 350 MW core (mHTGR-350). In the spring and summer of 2019, several experiments were conducted at HTTF providing a valuable source of gas-cooled reactor thermal hydraulics data. Idaho National Laboratory (INL), Oregon State University, Argonne National Laboratory, and Canadian Nuclear Laboratories have partnered to use this experimental data to develop a gas-cooled reactor thermal hydraulics benchmark led by INL under the auspices of the Advanced Reactor Technologies (ART) program. This report provides some context on the benchmark, HTTF, and previous Reactor Excursions and Leak Analysis Program (RELAP)5-3D modeling of HTTF. It also provides a draft of the benchmark specifications for the Depressurized Conduction Cooldown problem, which is the INL-led benchmark problem.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

A Strategy for Low-Mass Dark Matter Searches with Cryogenic Detectors in the SuperCDMS SNOLAB Facility

The SuperCDMS Collaboration is currently building SuperCDMS SNOLAB, a dark matter search focused on nucleon-coupled dark matter in the 1-5 GeV mass range. Looking to the future, the Collaboration has developed a set of experience-based upgrade scenarios, as well as novel directions, to extend the search for dark matter using the SuperCDMS technology in the SNOLAB facility. The experienced-based scenarios are forecasted to probe many square decades of unexplored dark matter parameter space below 5 GeV, covering over 6 decades in mass: 1-100 eV for dark photons and axion-like particles, 1-100 MeV for dark-photon-coupled light dark matter, and 0.05-5 GeV for nucleon-coupled dark matter. They will reach the neutrino fog in the 0.5-5 GeV mass range and test a variety of benchmark models and sharp targets. The novel directions involve greater departures from current SuperCDMS technology but promise even greater reach in the long run, and their development must begin now for them to be available in a timely fashion. The experienced-based upgrade scenarios rely mainly on dramatic improvements in detector performance based on demonstrated scaling laws and reasonable extrapolations of current performance. Importantly, these improvements in detector performance obviate significant reductions in background levels beyond current expectations for the SuperCDMS SNOLAB experiment. Given that the dominant limiting backgrounds for SuperCDMS SNOLAB are cosmogenically created radioisotopes in the detectors, likely amenable only to isotopic purification and an underground detector life-cycle from before crystal growth to detector testing, the potential cost and time savings are enormous and the necessary improvements much easier to prototype.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Detecting Process Equipment Failures Using Acoustic Data and Machine Learning

Nuclear power plant (NPP) process equipment such as fans, motors, valves, and pumps generate frequent or continuous noise, and deviations from the normal operational sounds made by this equipment can indicate potential issues. These deviations can be identified via automated acoustic anomaly detection, which involves using acoustic sensors (i.e., microphones) alongside detection algorithms to continuously monitor for changes in acoustic signatures. This task is made challenging by the substantial background noise that exists, such as operators opening and closing doors, manipulating valves, and conversing—in addition to typical plant noises. In collaboration with a nuclear power utility partner, this effort assessed the efficacy of acoustic anomaly detection when using a specific acoustic sensor that compresses data into a fixed set of features that are transferable over a standard Internet of Things communication protocol, thereby improving usability but potentially degrading detection performance. Two methods of performing automated acoustic anomaly detection were evaluated: one-class support vector machine (OC-SVM) and isolation forest (iForest). To enable the use of high-quality acoustic data encompassing both normal and anomalous conditions, the study utilized the publicly available Malfunctioning Industrial Machine Investigation and Inspection dataset, which includes real measured acoustic sensor data for a range of equipment types, model numbers, and signal-to-noise ratios (SNRs), along with a benchmark set of detection results. Using this dataset, the methods were tested and then compared against the benchmark results. The results indicated that although the specific acoustic sensor did not enable as rich a feature set extraction, the proposed methods with the limited feature set performed just as well. This provides solid justification for both the methods and the use of the proposed acoustic sensor.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

Engineering PHL7 for Improved Poly(Ethylene Terephthalate) Depolymerization via Rational Design and Directed Evolution

Enzymatic depolymerization of poly(ethylene terephthalate) (PET) has emerged as a promising approach for polyester recycling, and, to date, many natural and engineered PET hydrolase enzymes have been reported. For industrial use, PET hydrolases must achieve high depolymerization extent and exhibit excellent thermostability. Here, we engineered a natural PET hydrolase, Polyester Hydrolase Leipzig #7 (PHL7), through rational design and directed evolution using a high-throughput screening platform. Four new enzymes were engineered with enhanced properties compared with the parent enzyme, wild-type PHL7 (PHL7-WT), and other benchmark PET hydrolases, under the tested conditions. In bioreactors, the exemplary engineered enzyme, PHL7-Jemez, exhibited improved ability to depolymerize amorphous PET film compared with PHL7-WT at 2.9% and 20% substrate loadings, with 37% and 270% higher hydrolysis, respectively, after 48 h. This study develops several state-of-the-art PET hydrolases and demonstrates a directed evolution platform to engineer high-performance enzymes, which can accelerate enzyme discovery toward improved biocatalytic recycling.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Disjunctive linear separation conditions and mixed-integer formulations for aircraft conflict resolution

In this paper, we address the aircraft conflict resolution problem in air traffic control. We introduce new mixed-integer programming formulations for aircraft conflict resolution with speed, heading and altitude control which are based on disjunctive linear separation conditions. We first examine the two-dimensional aircraft conflict resolution problem with speed and heading control represented as continuous decision variables. We show that the proposed disjunctive linear separation conditions are equivalent to the classical nonlinear conditions for aircraft separation. Further, we characterise conflict-free trajectories based on aircraft velocity bounds and propose a simple pre-processing algorithm to identify aircraft pairs which are either always conflict-free, or which cannot be separated using speed and heading control only. We then incorporate altitude control and propose a lexicographic optimisation formulation that aims to minimise the number of flight level changes before resolving outstanding conflicts via two-dimensional velocity control. The proposed mixed-integer programming formulations are nonconvex, and we propose convex relaxations, decomposition methods and constraint generation algorithms to solve the two-dimensional and lexicographic optimisation formulations to guaranteed optimality. Numerical experiments on four types of conflict resolution benchmarking instances are conducted to test the performance of the proposed mixed-integer formulations. Further, the proposed method is compared against two benchmarks based on state-of-the-art approaches for the aircraft conflict resolution problem. Our numerical results show that the proposed method largely outperforms both benchmarks in terms of runtime and is able to solve significantly more instances to global optimality.

97 MATHEMATICS AND COMPUTING↗