Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Software Test Tool”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46

Application of reusable software components at the SEI

Robert Holibaugh of the Software Engineering Institute described a project which is studying the application of reusable software components. The primary goals are to gain practical experience with state-of-the-art reusable components, methods, and tools and to capture the lessons learned in the application of reuse technology. In addition the project will assess the impact of reuse on the software development process and products and will identify and validate the information that facilities software reuse during system development. The project includes two tasks - a reuse experiment and a redevelopment effort. The reuse experiment will define a life cycle and a methodology for reuse-based development, and define and implement a data collection mechanism for measuring the development. The redevelopment effort will construct a reuse test bed and will redevelop and realistically test subsystems from an embedded mission-critical real-time application. The reuse experiment will produce several products including a tested real-time application, reuse-based components and tools evaluation, a reuse-based development method, a framework for data collection, a framework for measuring productivity, and lessons learned data. Successful development with reusable components will require a rich set of components and an integrating methodology. The Tomahawk Land Attack Missile system is the application for the redevelopment effort.

Holibaugh, Robert↗

Distribution System Segmented Model Simplification with Independent Dynamically Changing End-Use Loads

With more emerging technologies in power distribution systems, to support quasi-static time-series (QSTS) simulation over short time frames, accurately modeling the dynamic behavior of end-use load under various conditions is im- portant. However, time and cost can be a burden for the QSTS simulation of thousands of houses representing dynamic load behaviors on large distribution feeders. There are two main contributions of this paper in order to reduce the QSTS simulation complexity. First, the paper presents a method to stochasti- cally model the dynamic behavior of end-use load for QSTS simulations. Second, using the stochastically modeled load behavior as an input to a distribution sys- tem, and building on our previous QSTS simulation simplification/acceleration method that relies on uniform load behavior, this paper presents a segment substitution approach for the simplification of distribution system models with independent dynamically changing end-use loads. Our numerical results show that compared to GridLAB-D, a software tool that simulates end-use load be- havior through physical models, the proposed stochastic modeling approach achieved an 89.3% simulation time reduction for end-use loads, while accurately capturing the temporally changing total load behavior. Moreover, the QSTS simulation of a large test distribution system through OpenDSS showed that the proposed simplification method decreased the feeder simulation time by a factor of three at less than 0.3% voltage error. Compared to our previous method with independent dynamic loads, the voltage error is reduced by a factor of four.

Power distribution system modeling, stochastic pro↗

miniVite: A Graph Analytics Benchmarking Tool for Massively Parallel Systems

Benchmarking of high performance computing sys-tems can help provide critical insights for ef?cient design of computing systems and software applications. Although a large number of tools for benchmarking exist, there is a lack of rep-resentative benchmarks for the class of irregular computations as exempli?ed by graph analytics. In this paper, we propose miniVite as a representative graph analytics benchmark tool to test a variety of distributed-memory systems. Graph clustering, popularly known as community detection, is a prototypical graph operation used in numerous scienti?c computing and analytics applications. The goal of clustering is to partition a graph into clusters (or communities) such that each cluster consists of vertices that are densely connected within the cluster and sparsely connected to the rest of the graph. Modu-larity optimization is a popular technique for identifying clusters in a graph. Ef?cient parallelization of modularity optimization-based algorithms is challenging. One successful approach was conceived in Vite, a distributed-memory implementation of the Louvain algorithm that incorporates several heuristics. We introduce miniVite as a representative but simpli?ed variant of Vite, to serve as a prototypical graph analytics benchmarking tool. Unlike other graph-based methods such as breadth-?rst search and betweenness centrality, miniVite represents highly complex computational patterns stressing a variety of system features, which can provide crucial insight for co-design of future computing systems.

community detection, parallel, MPI↗

Comprehensive Solutions for Integration of Solar Resources into Grid Operations

The need for proactive closed-loop integration of uncertainty information into system operations and probability-based controls is widely recognized, but rarely implemented in system operations. Proactive integration for this project means that the information concerning expected uncertainty ranges for net load and balancing requirements, including required balancing capacity, ramping and ramp duration characteristics, will be fed back into the generation commitment and dispatch algorithms to modify their performance so that potential shortages of these characteristics can be prevented. This basic, yet important, premise is the motivating factor for this project. The achieved project goal is to demonstrate the benefit of such a system. The project quantifies future uncertainties, predicts additional system balancing needs including the prediction intervals for capacity and ramping requirements of future dispatch intervals, evaluates the impacts of uncertainties on transmission including the risk of overloads and voltage problems, and explores opportunities for intra-hour generation adjustments helping to provide more flexibility for system operators. The resulting benefits culminate in more reliable grid operation in the face of increased system uncertainty and variability caused by solar power. The project identifies that solar power does not require special separate penetration level restrictions or penalization for its intermittency. Ultimately, the collective consideration of all sources of intermittency distributed over a wide area unified with the comprehensive evaluation of various elements of balancing process, i.e. capacity, ramping, and energy requirements, help system operators more robustly and effectively balance generation against load and interchange. This project showed that doing so can facilitate more solar and other renewable resources on the grid without compromising reliability and control performance. Efforts during the project included developing and integrating advanced probabilistic solar forecasts, including distributed PV forecasts, into closed –loop decision making processes. Additionally, new uncertainty quantifications methods and tools for the direct integration of uncertainty and variability information into grid operations at the transmission and distribution levels were developed and tested. During Phase 1, project work focused heavily on the design, development and demonstration of a set of processes and tools that could reliably and efficiently incorporate solar power into California’s grid operations. In Phase 2, connectivity between the ramping analysis tools and market applications software were completed, multiple dispatch scenarios demonstrated a successful reduction of overall uncertainty and an analysis to quantify increases in system operator reliability, and the transmission and distribution system uncertainty prediction tool was introduced to system operation engineers in a live webinar. The project met its goals, the experiments prove the advancements to methods and tools, when working together, are beneficial to not only the California Independent System Operator, but the benefits are transferable to other system operators in the United States.

14 SOLAR ENERGY↗

Fiscal Year 2023 Software Quality Assurance Activities for the ARC Software

The Argonne Reactor Code (ARC) software suite [1-17] has been developed by Argonne researchers for fast reactor design and analysis since the 1970s. With the ARC software suite, a user can quickly build a model of a proposed or existing fast spectrum reactor and carry out fuel cycle, nominal thermal analysis and flow requirements, and assess, as is appropriate, whether the core design and constraint system yield an acceptable mechanical behavior. For transient reactor analysis with SAS4A [18], the ARC software suite can be used to generate reactivity coefficients and kinetics parameters at any modeled fuel cycle time point which forms part of the input to SAS4A. The ARC suite was consistently being developed until the 1990s and followed a software QA program which was an appropriate standard for the time. In the 1990s, the DOE funding to fast reactor research and development was all but eliminated and the ARC software was put into maintenance mode. In the early 2000s, the software quality assurance (SQA) program for ARC was still in place to define an official version, but by 2005 it all but was abandoned as there were insufficient staff to fill the work roles. Since 2005, there has been a considerable increase in research and design work on fast spectrum reactors. The ARC software as a whole has since been exported to many universities and commercial companies and ANL support has been given to the various projects over the years [19-23]. Further, MC 2 -3, PERSENT, and DASSH were all developed after 2005 without any adherence to a software standard. In recent time, the DOE VTR project [22] paid for verification work to be done on the ARC software as part of the goal of making it NQA-1 complaint. The VTR project was not considered the appropriate pathway to fund and maintain a SQA program for the ARC software and while software developments (DASSH) were made and several manuals were updated and software verification work was carried out, the ARC software is not NQA-1 compliant. More recently the Advanced Reactor Development Program (ARDP [23]) has funded the creation of manuals for some ARC utility programs and funded additional software verification work on DIF3D [6, 7] and MC 2 -3 [2-5] for the purpose of commercial grade dedication. Because of the VTR and ARDP projects, software verification work was completed on MC 2 -3 and DIF3D, and detailed reports were created for each piece of software, which discuss the inputs and outputs from the codes that are covered by the verification work and link various analytic, code-to-code, and hand calculation based verification work presented in the report with verification test problems provided with the software. This is a key part of the commercial grade dedication work and constitutes the bulk of the cost to get the ARC software to commercial grade. The ARC software suite is a valuable asset as a fast reactor design and analysis tool set that has been reasonably well verified and validated with various fast reactor benchmark problems and experiments over decades. Some or all of the ARC software suite has been utilized for designing the IFR [20], PGSFR [21], VTR [22], and Natrium [23] reactors and we can expect it to continue to be used for advanced fast reactor design and/or confirmatory calculation purposes in the future. Due to increased interest by commercial companies and regulatory bodies, it is becoming more important to make the ARC software suite complete and ready-to-use in terms of its SQA pedigree and commercial grade dedication needs. This report discusses the achievements made towards building a new SQA program for the ARC software and dealing with outstanding identified QA gaps.

97 MATHEMATICS AND COMPUTING↗

Detector Simulation Challenges for Future Accelerator Experiments

Detector simulation is a key component for studies on prospective future high-energy colliders, the design, optimization, testing and operation of particle physics experiments, and the analysis of the data collected to perform physics measurements. This review starts from the current state of the art technology applied to detector simulation in high-energy physics and elaborates on the evolution of software tools developed to address the challenges posed by future accelerator programs beyond the HL-LHC era, into the 2030–2050 period. New accelerator, detector, and computing technologies set the stage for an exercise in how detector simulation will serve the needs of the high-energy physics programs of the mid 21st century, and its potential impact on other research domains.

43 PARTICLE ACCELERATORS↗

CASTLE: Conflict Analysis Strategy Testing Laboratory Environment v.1.0.0

SAND2024-01743O The Conflict Analysis Strategy Testing Laboratory Environment (CASTLE) is a software framework that enables and simplifies building a novel, turn-based strategy game in which it can define its own rules, maps, pieces, and interactions. The software is for novice to experienced programmers with some knowledge of Unity3D, a tool used in game production. CASTLE includes a library of common game mechanics used for strategic wargames and traditional board games, such as cards, tokens, dice, and grid maps. It follows design principles popularized by the video game industry and uses singletons for managing portions of the code. CASTLE builds on Unity's component-based design and can respond to engine events during execution. Among the numerous user-friendly features: Build games quickly and cost-effectively Network in real-time and apply data to new games developed on the framework Host multiple participants online Connect rule- or machine learning-based agents to a CASTLE game to serve as opponents or to simulate games Collect data collection from players and in-game behaviors Create a survey to gather demographics or opinions from players Store data locally or save it to an external database through Representational State Transfer (REST) functions CASTLE, which was prototyped using Microsoft Azure, is also designed for easily distributing online games using popular cloud services. The multiplayer functionality includes an agent interface, allowing developers to construct AI players that can substitute for humans in any of the games. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Fabian, Nathan↗

Material Accountancy Performance Indicator Toolkit

The MAPIT - Material Accountancy Performance Indicator Toolkit includes statistical tests and calculations based on open literature that calculate safeguards metrics. The tool can be used with user supplied datasets or with the several included datasets which are generated from models that are maintained at Sandia National Laboratories. The models are not included with the software. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-0467 O

Shoman, Nathan↗

AMSS

SAND2022-12902 O The Automated Method Selection Software (AMSS) is a chromatographic prediction tool based on the previously published BioCompound Machine Learning (BCML) tool. AMSS uses PubChem ID as input for molecule information such as SMILES and physical and chemical properties. The model is used to predict the best chromatography analysis method for testing compounds. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Poorey, Kunal↗

Validation testing of Canberra-Obayashi TruckScan calculation method - 15409

Canberra Industries, Inc. [CI] has designed a new truck monitoring system 'TruckScan' for Interim Storage Facilities [ISF] in Japan. CI and Canberra Japan K.K. [CJKK] have performed validation testing of it with Obayashi Corporation at a temporary storage area in Fukushima prefecture. Decontaminated waste was put into flexible containers called Super Sacks [SS]. The customer desires to quantify the Cs-137 and Cs-134 content of each individual SS, while a group of them are on a truck. This would be done when leaving a temporary waste consolidation area and entering an ISF. The content of SSs is some combination of sand, soil, and vegetation with densities ranging from 0.3 g/cc - 1.6 g/cc. The typical weight of the trucks will be approximately 10 tons, but can vary between 4 and 20 tons. The system must be sensitive enough to detect 100 Bq/kg in 10 - 30 seconds but still have enough dynamic range to measure 1,000,000 Bq/kg material. The system will be operated in an outdoor environment. The desire to separately quantify Cs-137 and Cs-134 favors the use of a spectroscopic system as a solution. The full-scale TruckScan will consist of eight 3x3' NaI detectors, each in a lead shield with a collimated view of the truck. Four detectors are on each side of the truck, at about 1 meter from the truck, spaced at equal distances. These NaI detectors and collimators were calibrated by In Situ Object Counting System (ISOCS) mathematical efficiency calculation tool. The truck stops in-between the two sets of detectors for the short measurement period - typically 15 seconds. The special software performs gamma spectroscopy on each of the 8 spectra, and then decodes the results to determine the activity in each of the 8 [typically] SSs using a Maximum Entropy Analysis Method. Validation testing was done by using SSs filled with known material types and known concentrations of material. The results of the TruckScan assay indicate good accuracy for the wide range of conditions. In spite of the conditions being more severe than normal operations, the combined standard deviation was 20.1% for the 6 detector version and 16.6% for the 8 detector version TruckScan. When the TruckScan results were compared to the known concentrations, there was minimal bias and good correlation [y = 1.0029x, R{sup 2} = 0.914]. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Laboratory test methodology for evaluating the effects of electromagnetic disturbances on fault-tolerant control systems

Control systems for advanced aircraft, especially those with relaxed static stability, will be critical to flight and will, therefore, have very high reliability specifications which must be met for adverse as well as nominal operating conditions. Adverse conditions can result from electromagnetic disturbances caused by lightning, high energy radio frequency transmitters, and nuclear electromagnetic pulses. Tools and techniques must be developed to verify the integrity of the control system in adverse operating conditions. The most difficult and illusive perturbations to computer based control systems caused by an electromagnetic environment (EME) are functional error modes that involve no component damage. These error modes are collectively known as upset, can occur simultaneously in all of the channels of a redundant control system, and are software dependent. A methodology is presented for performing upset tests on a multichannel control system and considerations are discussed for the design of upset tests to be conducted in the lab on fault tolerant control systems operating in a closed loop with a simulated plant.

Belcastro, Celeste M.↗

SU-F-P-54: Guidelines to Check Image Registration QA of a Clinical Deformation Registration Software: A Single Institution Preliminary Study

Purpose: The objective of this study is to verify and analyze the accuracy of a clinical deformable image registration (DIR) software. Methods: To test clinical DIR software qualitatively and quantitatively, we focused on lung radiotherapy and analyzed a single (Lung) patient CT scan. Artificial anatomical changes were applied to account for daily variations during the course of treatment including the planning target volume (PTV) and organs at risk (OAR). The primary CT (pCT) and the structure set (pST) was deformed with commercial tool (ImSimQA-Oncology Systems Limited) and after artificial deformation (dCT and dST) sent to another commercial tool (VelocityAI-Varian Medical Systems). In Velocity, the deformed CT and structures (dCT and dST) were inversely deformed back to original primary CT (dbpCT and dbpST). We compared the dbpST and pST structure sets using similarity metrics. Furthermore, a binary deformation field vector (BDF) was created and sent to ImSimQA software for comparison with known “ground truth” deformation vector fields (DVF). Results: An image similarity comparison was made by using “ground truth” DVF and “deformed output” BDF with an output of normalized “cross correlation (CC)” and “mutual information (MI)” in ImSimQA software. Results for the lung case were MI=0.66 and CC=0.99. The artificial structure deformation in both pST and dbpST was analyzed using DICE coefficient, mean distance to conformity (MDC) and deformation field error volume histogram (DFEVH) by comparing them before and after inverse deformation. We have noticed inadequate structure match for CTV, ITV and PTV due to close proximity of heart and overall affected by lung expansion. Conclusion: We have seen similarity between pCT and dbpCT but not so well between pST and dbpST, because of inadequate structure deformation in clinical DIR system. This system based quality assurance test will prepare us for adopting the guidelines of upcoming AAPM task group 132 protocol.

60 APPLIED LIFE SCIENCES↗

Robotics Verification and Validation Strategies for Perseverance Rover Sampling and Caching

The Mars 2020 Sampling and Caching Subsystem(SCS) is the most complex robotic system ever fielded on a MarsRover. It includes a 5 degree-of-freedom Robotic Arm, coringdrill, gas Dust Removal Tool, interfaces for two turret-mountedinstruments, and an Adaptive Caching Assembly (ACA). TheACA is itself a complex robotic system, containing hardware tosupport docking and bit exchange, a 3 degree-of-freedomSample Handling Assembly for manipulating sample tubes,storage for several drill bits and sample tubes, and mechanismsto support observing and sealing samples collected by the drill.To successfully verify and validate the SCS hardware andsoftware and its integration with the Mars 2020 flight systemseveral key strategies were employed.The SCS Verification and Validation (V&V) program utilizedmultiple test venues with tiered levels of fidelity. These includedsimulation and visualization software environments, low fidelitydevelopment testbeds, testbeds with high fidelity SCS hardwareand commercial off-the shelf avionics, integrated systemtestbeds with flight-like avionics, and environmental testbedscapable of simulating Martian surface temperature andpressure. Multiple units of each SCS hardware componentmoved fluidly between test venues to accomplish myriadstandalone and coordinated test objectives. Test preparationand executions were performed by a diverse team of engineerswith training and technical ownership tailored for individualexperience and role. Despite significant differences between testvenues, the SCS V&V team established efficient and consistentprocesses and tools for procedure development, test execution,and data review that enabled personnel, as well as technicalproducts such as sequences and parameter configurations, toflow between venues effectively. A series of benchmark testsprovided evidence of performance consistency as elements weretransferred between venues and as system capability evolved.This paper provides an overview of the SCS V&V program andexplores several overarching strategies that enabled successfuloperation in the face of unprecedented complexity. Keyoutcomes of the SCS validation effort are summarized, alongwith lessons learned and beneficial integrations of validationtool and process innovations into Mars surface operations.

Brooks, Sawyer↗

SOWFA Super-Controller: A High-Fidelity Tool for Evaluating Wind Plant Control Approaches

This paper presents a new tool for testing wind plant controllers in the Simulator for Offshore Wind Farm Applications (SOWFA). SOWFA is a high-fidelity simulator for the interaction between wind turbine dynamics and the fluid flow in a wind plant. The new super-controller testing environment in SOWFA allows for the implementation of the majority of the wind plant control strategies proposed in the literature.

17 WIND ENERGY↗

pyEF: A Python Framework for QM and QM/MM Atom-Wise Electric Field Analysis

We introduce pyEF, a software package for computing molecular electric fields, electrostatic interaction energies, and electrostatic potentials from quantum mechanical (QM) atom-centered multipole expansions with atom-wise decomposable contributions. We demonstrate the computational efficiency and accuracy of this QM-derived electric field evaluation tool through several tests. To assess the influence of the underlying QM method and charge partitioning scheme on these electrostatic quantities, we analyze over 250 configurations of an acetone solute molecule in five solvents of variable polarity. We find that electric field calculations are highly sensitive to the choice of charge partitioning method. Even among real-space charge schemes, acetone Stark tuning rates differ by up to a factor of 2. Benchmarking computed solvent dipole moments against experimental bulk values, we conclude that the CM5, ADCH, and Hirshfeld-I charge schemes most reliably capture solvent electrostatics and therefore provide a more faithful foundation for computing electric fields. When constructed from these real-space charges, electric fields are nearly insensitive to basis set size and monotonically increase in magnitude with higher Fock exchange. We also demonstrate efficient convergence of QM electrostatics when more distant molecules are represented solely by MM point charges, reducing computational overhead. Leveraging these findings, we demonstrate the use of pyEF to deduce environmental effects on a transition metal complex from a Ga 4 L 6 12– nanocage and quantify the dominant role of organic linkers in orchestrating electrostatic preorganization.

electric fields↗

Continuous Integration, In-Code Documentation, and Automation for Nuclear Quality Assurance Conformance

The Multiphysics Object Oriented Simulation Environment (MOOSE) is an open-source, finite element framework for solving highly coupled sets of nonlinear equations. The development of the framework and applications occurs concurrently using an agile, continuous-integration software package. Included in the framework is an in-code, extensible documentation system. Using these two tools in union with the repository management tools GitHub and GitLab, a software quality plan was created and followed such that MOOSE and a MOOSE-based application (BISON) have been shown to meet the American Society of Mechanical Engineers’ Nuclear Quality Assurance-1 standard. The approach relies heavily on automation for both testing and documentation. The resulting effort demonstrates that a rigorous software quality plan may be implemented that incurs a minimal impact on day-to-day development of the software, satisfying the stringent guidelines necessary to operate the software in a safety function within a nuclear facility.

97 MATHEMATICS AND COMPUTING↗

Rapid development of the X-31 simulation to support flight-testing

The X-31 Enhanced Fighter Maneuverability Program has been recognized to form the International Test Organization, with the NASA Dryden Flight Research Facility (NASA-Dryden) as the responsible test organization. The two X-31 research aircraft and engineering support personnel were colocated at NASA-Dryden, with flight test operations beginning in Apr. 1992. Therefore, rapid development of a hardware-in-the-loop simulation was needed to support the flight test operations at NASA-Dryden, and to perform verification and validation of flight control software. The X-31 simulation system requirements, distributed simulation system architecture, simulation components math models to the visual system, and the advanced capabilities the X-31 simulation provides. In addition, unique software tools and the methods used to rapidly develop this simulation system will be highlighted.

Mackall, Dale↗

Rapid development of the X-31 simulation to support flight-testing

The X-31 Enhanced Fighter Maneuverability Program has been recognized to form the International Test Organization, with the NASA Dryden Flight Research Facility (NASA-Dryden) as the responsible test organization. The two X-31 research aircraft and engineering support personnel were colocated at NASA-Dryden, with flight test operations beginning in Apr. 1992. Therefore, rapid development of a hardware-in-the-loop simulation was needed to support the flight test operations at NASA-Dryden, and to perform verification and validation of flight control software. The X-31 simulation system requirements, distributed simulation system architecture, simulation components math models to the visual system, and the advanced capabilities the X-31 simulation provides. In addition, unique software tools and the methods used to rapidly develop this simulation system will be highlighted.

Mackall, Dale↗