Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

An Evaluation of the ASM2000 software

The ASM2000 software is based on the NetCAM3 software (currently installed at RLUOB) with the main difference being enhanced multi-CAM head support. Therefore, as the NetCAM3 software has been previously tested, the current evaluation concentrated on performance related to multi-head operation. In addition, bug fixes and improvements to prior versions of the NetCAM3 code were incorporated into the ASM2000 V2.0.0 software package.

97 MATHEMATICS AND COMPUTING↗

A Python Object Oriented & Data Driven Testing Infrastructure for an Environmental Information Management System

Software systems such as TEIMS, the Taurus Environmental Information Management System evolve in size and complexity making manual testing no longer sufficient or feasible, becoming a performance and quality risk. TEIMS is a system of systems comprised of dynamic, data-driven web-based applications that support planning, operational monitoring, data collection, sample management, reporting and, scientific findings for LLNL’s Environmental Restoration Department (ERD). Selenium’s Integrated Development Environment (IDE) was previously used to support TEIMS testing. An approach with limited flexibility for future software development. As a student intern I was assigned the task to develop an object oriented testing infrastructure for TEIMS. A critical resource in this approach is the development of a catalogue of page elements. To accomplish this I created a comma separated value file, which is comprised of key identifiers based on page elements. Using the Python language and the Selenium Web driver I then developed the object mapping structure, which serves as a wrapper for exposure of each element allowing interaction with the tests and test suites. A master driver “Module” allowed the web driver to be shared among the objects and tests. I then developed a web-based graphical user interface (GUI) for test history, metrics and searching. This data driven approach provides flexibility and extensibility, allowing changes to the structure to be performed in a single place. The result is an object oriented/data driven infrastructure that allows tests to be simpler, flexible, and maintainable, with searchable results.

97 MATHEMATICS AND COMPUTING↗

Bayes Inference Engine (BIE) Test Plan: TiltConeBeamFVT

The Bayes Inference Engine (BIE) is a general software tool intended to be used primarily in the analysis of radiographic data for density. An analysis is set up in the BIE by representing the problem as a collection of modules called glyphs. All glyphs to be used in creating the forward model of the experiment will be tested separately for a range of inputs. The testing of the adjoint code is performed in a separate document.

97 MATHEMATICS AND COMPUTING↗

Thermal properties and Thermodynamic Equilibrium Modeling of Cementitious Waste Forms

INTRODUCTION Cementitious matrices are used in the US DOE complex and worldwide to solidify aqueous radioactive, hazardous, mixed salt solutions, and sludges to meet low-level radioactive waste (LLW) disposal requirements. Blended formulations are used for most waste form mix designs. Substitution of pozzolans for Class F fly ash has the potential to alter the processing properties and stabilization properties of the resulting waste forms because they add chemical and mineralogical complexity to the final material. The Savannah River Site saltstone waste form was selected as the test case material. CemGEMS software was chosen as the model for predicting hydrated cementitious phases assemblages and phase evolution. Isothermal calorimetry was used as the method for evaluating processing and properties of fresh, uncured waste forms. The test cases consisted of reference case saltstone, and five natural pozzolan substituted saltstone mixes.

Bustamante, Michael E. [Savannah River National La↗

Oklo Sponsored Testing using PELICAN: System, Subchannel, and High-Fidelity Software Validation (Final CRADA Report)

Argonne National Laboratory (the Contractor), located in Lemont IL, and Oklo, Inc., (the Participant), headquartered in Santa Clara, CA, propose to enter into a Cooperative Research and Development Agreement (CRADA) to perform a gap analysis of thermal hydraulic data, perform prototypical fuel assembly pressure drop and cavitation model validation, generate the experimental data as well as the corresponding uncertainties for this matrix and, finally, develop the validation models with the Argonne system level code SAS4A/SASSYS-1, the Argonne subchannel analysis code DASSH, the Argonne high fidelity code Nek5000, and/or the Idaho National Laboratory code Pronghorn Subchannel. The work outlined below will significantly improve the experimental and validation database currently available for liquid metal fast reactors, thus making it a viable part of a comprehensive reactor design and licensing suite to be used by the participant.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Standards, dissemination, and best practices in systems biology

In this study, the reproducibility of scientific research is crucial to the success of the scientific method. Here, we review the current best practices when publishing mechanistic models in systems biology. We recommend, where possible, to use software engineering strategies such as testing, verification, validation, documentation, versioning, iterative development, and continuous integration. In addition, adhering to the Findable, Accessible, Interoperable, and Reusable modeling principles allows other scientists to collaborate and build off of each other’s work. Existing standards such as Systems Biology Markup Language, CellML, or Simulation Experiment Description Markup Language can greatly improve the likelihood that a published model is reproducible, especially if such models are deposited in well-established model repositories. Where models are published in executable programming languages, the source code and their data should be published as open-source in public code repositories together with any documentation and testing code. For complex models, we recommend container-based solutions where any software dependencies and the run-time context can be easily replicated.

59 BASIC BIOLOGICAL SCIENCES↗

UnitTesterSG

UnitTesterSG is a unit testing framework that is designed for nested and/or scientific/engineering data structures. It is designed primarily for testing the outputs if a single function or simulation run by storing the expected results file such that comparisons to the stored output can be made with unit tests after the function or software has been edited. This module is designed to be able to unit test arrays, strings, as well as nested and deeply nested objects (including arrays with tuples inside, arrays with strings inside, etc.). The module is compatable with nested objects, and thus has a dependency on the nestedObjectsFunctions module within. The module takes *calculated* results and then compares them to *expected* results. If no expected results are available (or they do not match the calculated results) the module then offers to store the calculated results as expected results for next time.

Savara, Aditya [Oak Ridge National Lab. (ORNL), Oa↗

Influence of Fuel Properties on Gasoline Direct Injection Particulate Matter Emissions Over First 200 s of World-Harmonized Light-Duty Test Procedure Using an Engine Dynamometer and Novel “Virtual Drivetrain” Software

Abstract The influence of fuel properties on particulate matter (PM) emissions from a catalytic gasoline particulate filter (GPF) equipped gasoline direct injection (GDI) engine was investigated using novel “virtual drivetrain” software and an engine mated to an engine dynamometer. The virtual drivetrain software was developed in labview to operate the engine on an engine dynamometer as if it were in a vehicle undergoing a driving cycle. The software uses a physics-based approach to determine vehicle acceleration and speed based on engine load and a programed “shift” schedule to control engine speed. The software uses a control algorithm to modulate engine load and braking to match a calculated vehicle speed with the prescribed speed trace of the driving cycle of choice. The first 200 s of the World-harmonized Light-duty Test Procedure (WLTP) driving cycle was tested using six different fuel formulations of varying volatility, aromaticity, and ethanol concentration. The first 200 s of the WLTP was chosen as the test condition because it is the most problematic section of the driving cycle for controlling PM emissions due to the cold start and cold drive off. It was found that there was a strong correlation between aromaticity of the fuel and the engine-out PM emissions, with the highest emitting fuel producing more than double the mass emissions of the low PM production fuel. However, the post-GPF PM emissions depended greatly on the soot loading state of the GPF. The fuel with the highest engine-out PM emissions produced comparable post-GPF emissions to the lowest PM producing fuel over the driving cycle when the GPF was loaded over three cycles with the respective fuels. These results demonstrate the importance of GPF loading state when aftertreatment systems are used for PM reduction. It also shows that GPF control may be more important than fuel properties, and that regulatory compliance for PM can be achieved with proper GPF control calibration irrespective of fuel type.

Energy & Fuels↗

Julienne v1.0.0

Julienne is a compiler-portable unit-testing framework for Fortran software projects, including those that use the parallel/accelerator-programming features of Fortran 2023. Julienne achieves portability across compilers through minimalism and isolation. The minimal design ensures that Julienne uses only features supported by the majority of Fortran compilers. The isolation through zero dependencies ensures that no other projects block Julienne from building with a particular compiler. Julienne also contains with additional services that support its unit-testing code. These include functions for manipulating strings, command lines and input/output format strings; and a user-defined collective subroutine for verifying that all processes pass a test in parallel testing. Julienne's name derives from the term for vegetables sliced into thin strings: julienne vegetables. Julienne captures the authors' most frequently used thin slice of the Veggies and Sourcery software repositories while avoiding certain compiler limitations of the those two packages.

Rouson, Damian↗

SU-E-T-27: A Tool for Routine Quality Assurance of Radiotherapy Dose Calculation Software

Purpose: Dose calculation software is thoroughly evaluated when it is commissioned; however, evaluation of periodic software updates is typically limited in scope due to staffing constraints and the need to quickly return the treatment planning system to clinical service. We developed a tool for quickly and comprehensively testing and documenting dose calculation software against measured data. Methods: A tool was developed using MatLab (The MathWorks, Natick, MA) for evaluation of dose calculation algorithms against measured data. Inputs to the tool are measured data, reference DICOM RT PLAN files describing the measurements, and dose calculations in DICOM format. The tool consists of a collection of extensible modules that can perform analysis of point dose, depth dose curves, and profiles using dose difference, distance-to-agreement, and the gamma-index. Each module generates a report subsection that is incorporated into a master template, which is converted to final form in portable document format (PDF). Results: After each change to the treatment planning system, a report can be generated in approximately 90 minutes. The tool has been in use for more than 5 years, spanning 5 versions of the eMC and 4 versions of the AAA. We have detected changes to the algorithms that affected clinical practice once during this period. Conclusion: Our tool provides an efficient method for quality assurance of dose calculation software, providing a complete set of tests for an update. Future work includes the addition of plan level tests, allowing incorporation of, for example, the TG-119 test suite for IMRT, and integration with the treatment planning system via an application programming interface. Integration with the planning system will permit fully-automated testing and reporting at scheduled intervals.

07 ISOTOPE AND RADIATION SOURCES↗

OTERR User Manual

OTERR (Optimization of TEst Reactor Reloading) is a software tool created to assist in the determination of optimal fuel reloading patterns for test reactors. The intended application is for the Versatile Test Reactor (VTR) program, but it provides functions that could be useful for analysis and optimization of many types of fast reactors. OTERR does not perform neutron/gamma transport, heat transfer, thermal hydraulics, or depletion calculations. Instead, it acts as a wrapper around codes that provide these capabilities, with a native genetic algorithm optimization capability. At this time, wrapping is only implemented for Argonne Reactor Computation (ARC) codes DIF3D, REBUS, and GAMSOR, and SE2-ANL. OTERR also has capabilities to facilitate input creation for DASSH, a thermal hydraulics code similar to SE2- ANL being developed for the VTR program.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Software Quality Assurance for the MOOSE-Based Open-Source Multiphysics Code Cardinal - An Expanded CI Testing Suite

Cardinal is a wrapping of the GPU-oriented spectral element Computational Fluid Dynamics (CFD) code NekRS and the Monte Carlo particle transport code OpenMC within the Multiphysics Object-Oriented Simulation Environment (MOOSE). Cardinal provides high-resolution thermal-hydraulics and/or radiation transport feedback to MOOSE multiphysics simulations. Multiphysics feedback is implemented in a geometry-agnostic manner which eliminates the need for rigid one-to-one mappings. A generic data transfer implementation also allows NekRS and OpenMC to couple to any MOOSE application, enabling a broad set of multiphysics capabilities. Cardinal simulations can also leverage combinations of MPI, OpenMP, and GPU resources. Cardinal continuous development and improvement efforts have led to the software being considered as a high-fidelity design and licensing tool for key areas of nuclear reactor relevant physics, including neutron transport, fluid flow, heat transfer, and mechanical processes. The fast development and expansion of the software from a pure R&D framework towards its application in the nuclear industry and regulation require a focus on developing, enhancing and, maintaining Cardinal’s software quality through strict adherence to a Software Quality Assurance (SQA) framework and SQA program. To facilitate compliance with SQA standards, the Cardinal SQA Program has been initiated during Fiscal Year 2023 (FY23). During the development of the Cardinal SQA Program, multiple gaps have been identified. These gaps are primarily related to model verification and code pedigree as they relate to the use of Cardinal as a safety analysis tool. These gaps have been captured in a report published in 2023. A second report highlighted the progress made during Fiscal Year 2024 (FY24) and described Argonne’s effort to document and integrate software verification within Cardinal’s software development process. This report documents a snapshot of the verification test cases currently available for Cardinal and NekRS in their assimilation into a Continuous Integration (CI) platform. Following the CI practice permits the integrating of source code changes frequently and ensuring that the integrated codebase clears the verification testing for the software. It should be noted that the SQA program itself, including the program plans, procedures, configuration management, and testing strategies, need to be developed in a future step of this task.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A Large-Scale Analysis to Optimize the Control and V2V Communication Protocols for CDA Agreement-Seeking Cooperation

Cooperative driving automation (CDA) Class C, agreement-seeking cooperation, is an innovative and practical solution that can promote cooperation among general passenger vehicles on the road. However, more comprehensive studies are needed before establishing the standard protocols of agreement-seeking cooperation, such as communication frequency and the duration of cooperation. Here, this article presents an initiative study on the impacts of communication capabilities on agreement-seeking cooperation. Through a large-scale analysis by regulating vehicle-to-vehicle (V2V) communication metrics, this work suggests desirable system parameters that can maximize the benefits of cooperation and ensure reliable operability while avoiding exhaustive communication loads. As the first step, an example agreement-seeking cooperation system is created for a car-following scenario, including decision-making and control algorithms for autonomous vehicles. Then, software-in-the-loop tests explore the performance of the developed system as it encounters various communication risks, such as latency and message packet drops. The system performance metrics are evaluated from various angles, including the time consumed for the agreement-seeking process, cooperation ratio, and the ratio of faulty cooperation. Energy saving from the cooperation is assessed by using simulation software that can run multiple high-fidelity vehicle models simultaneously. Based on the analyses, this article suggests the V2V communication requirements for the reliable operation of CDA agreement-seeking, which can be referred to when developing the standard protocols of agreement-seeking cooperation.

42 ENGINEERING↗

Methods, software and datasets to verify DVH calculations against analytical values: Twenty years late(r)

Purpose: The authors designed data, methods, and metrics that can serve as a standard, independent of any software package, to evaluate dose-volume histogram (DVH) calculation accuracy and detect limitations. The authors use simple geometrical objects at different orientations combined with dose grids of varying spatial resolution with linear 1D dose gradients; when combined, ground truth DVH curves can be calculated analytically in closed form to serve as the absolute standards. Methods: DICOM RT structure sets containing a small sphere, cylinder, and cone were created programmatically with axial plane spacing varying from 0.2 to 3 mm. Cylinders and cones were modeled in two different orientations with respect to the IEC 1217 Y axis. The contours were designed to stringently but methodically test voxelation methods required for DVH. Synthetic RT dose files were generated with 1D linear dose gradient and with grid resolution varying from 0.4 to 3 mm. Two commercial DVH algorithms—PINNACLE (Philips Radiation Oncology Systems) and PlanIQ (Sun Nuclear Corp.)—were tested against analytical values using custom, noncommercial analysis software. In Test 1, axial contour spacing was constant at 0.2 mm while dose grid resolution varied. In Tests 2 and 3, the dose grid resolution was matched to varying subsampled axial contours with spacing of 1, 2, and 3 mm, and difference analysis and metrics were employed: (1) histograms of the accuracy of various DVH parameters (total volume, D{sub max}, D{sub min}, and doses to % volume: D99, D95, D5, D1, D0.03 cm{sup 3}) and (2) volume errors extracted along the DVH curves were generated and summarized in tabular and graphical forms. Results: In Test 1, PINNACLE produced 52 deviations (15%) while PlanIQ produced 5 (1.5%). In Test 2, PINNACLE and PlanIQ differed from analytical by >3% in 93 (36%) and 18 (7%) times, respectively. Excluding D{sub min} and D{sub max} as least clinically relevant would result in 32 (15%) vs 5 (2%) scored deviations for PINNACLE vs PlanIQ in Test 1, while Test 2 would yield 53 (25%) vs 17 (8%). In Test 3, statistical analyses of volume errors extracted continuously along the curves show PINNACLE to have more errors and higher variability (relative to PlanIQ), primarily due to PINNACLE’s lack of sufficient 3D grid supersampling. Another major driver for PINNACLE errors is an inconsistency in implementation of the “end-capping”; the additional volume resulting from expanding superior and inferior contours halfway to the next slice is included in the total volume calculation, but dose voxels in this expanded volume are excluded from the DVH. PlanIQ had fewer deviations, and most were associated with a rotated cylinder modeled by rectangular axial contours; for coarser axial spacing, the limited number of cross-sectional rectangles hinders the ability to render the true structure volume. Conclusions: The method is applicable to any DVH-calculating software capable of importing DICOM RT structure set and dose objects (the authors’ examples are available for download). It includes a collection of tests that probe the design of the DVH algorithm, measure its accuracy, and identify failure modes. Merits and applicability of each test are discussed.

60 APPLIED LIFE SCIENCES↗

In Situ Data Infrastructure for Scientific Unit Testing Platform

Testing is a significant software development process for the management of software systems and scientific code. However, as the complexity of scientific codes increases, extra checks are needed to monitor impacts to dependent models and to verify system constraints. The software complexity also impedes the efforts of module developers and software engineers to rapidly develop and extend their code. Recently, we have developed an automatic methodology and prototype platform to facilitate scientific verification of individual functions within complex scientific codes. With this system, the scientific module builders are able to track variables conveniently in one module or track variables’ changes among different modules. In this paper, we present a procedure for automatic unit testing generation. For the interest of a general audience of this conference, we are emphasizing the technical details of integrating the In Situ data infrastructure into our platform. At the end of this paper, we have included an implementation of unit testing for the ACME Land Model (ALM) to demonstrate the usefulness and correctness of the platform. We have also used single- and multipoint checks to demonstrate the efficient variable tracking capability of this platform.

97 MATHEMATICS AND COMPUTING↗

TU-AB-202-03: Prediction of PET Transfer Uncertainty by DIR Error Estimating Software, AUTODIRECT

Purpose: Deformable image registration (DIR) is a powerful tool, but DIR errors can adversely affect its clinical applications. To estimate voxel-specific DIR uncertainty, a software tool, called AUTODIRECT (automated DIR evaluation of confidence tool), has been developed and validated. This work tests the ability of this software to predict uncertainty for the transfer of standard uptake values (SUV) from positron-emission tomography (PET) with DIR. Methods: Virtual phantoms are used for this study. Each phantom has a planning computed tomography (CT) image and a diagnostic PET-CT image set. A deformation was digitally applied to the diagnostic CT to create the planning CT image and establish a known deformation between the images. One lung and three rectum patient datasets were employed to create the virtual phantoms. Both of these sites have difficult deformation scenarios associated with them, which can affect DIR accuracy (lung tissue sliding and changes in rectal filling). The virtual phantoms were created to simulate these scenarios by introducing discontinuities in the deformation field at the lung rectum border. The DIR algorithm from Plastimatch software was applied to these phantoms. The SUV mapping errors from the DIR were then compared to that predicted by AUTODIRECT. Results: The SUV error distributions closely followed the AUTODIRECT predicted error distribution for the 4 test cases. The minimum and maximum PET SUVs were produced from AUTODIRECT at 95% confidence interval before applying gradient-based SUV segmentation for each of these volumes. Notably, 93.5% of the target volume warped by the true deformation was included within the AUTODIRECT-predicted maximum SUV volume after the segmentation, while 78.9% of the target volume was within the target volume warped by Plastimatch. Conclusion: The AUTODIRECT framework is able to predict PET transfer uncertainty caused by DIR, which enables an understanding of the associated target volume uncertainty.

60 APPLIED LIFE SCIENCES↗