Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

SAS4A/SASSYS-1 VERIFICATION AND VALIDATION TEST SUITE

SAS4A/SASSYS-1 is a safety analysis software package already under copyright by Argonne National Laboratory. There is interest from third-party users of the software to gain access to our internal verification and validation test suite. Therefore we are seeking to establish copyright over the test suite, which includes input files and reference results for hundreds of test cases that demonstrate functionality of the software. Making the test suite available to licensed users allows them to utilize the broad range of tests for commercial grade dedication activities needed to qualify the SAS4A/SASSYS-1 software under their own Software Quality Assurance program for use in a regulatory environment (e.g. with the Nuclear Regulatory Commission).

FANNING, THOMAS↗

EPICS Deployment at Fermilab

Fermilab has traditionally not been an EPICS house, as such expertise in EPICS is limited and scattered. However, PIP-II will be using EPICS for its control system. Furthermore, when PIP-II is operating, it must to interface with the existing, though modernized (see ACORN) legacy control system. We have developed and deployed a software pipeline that addresses these needs and presents to developers a tested and robust software framework, including template IOCs from which new developers can quickly gain experience. In this presentation, we will discuss the motivation for this work, the implementation of a continuous integration/continuous deployment pipeline, testing, template IOCs, and the deployment of user applications. We will also discuss how this is used with the current PIP-II teststand and lessons learned.

43 PARTICLE ACCELERATORS↗

Challenges and Strategies for Testing Automation Practices at Sandia National Laboratories

Sandia National Laboratories is a premier United States national security laboratory which develops science-based technologies in areas such as nuclear deterrence, energy production, and climate change. Computing plays a key role in its diverse missions, and within that environment, Research Software Engineers (RSEs) and other scientific software developers utilize testing automation to ensure quality and maintainability of their work. We conducted a Participatory Action Research study to explore the challenges and strategies for testing automation through the lens of academic literature. Through the experiences collected and comparison with open literature, we identify these challenges in testing automation and then present strategies for mitigation grounded in evidence-based practice and experience reports that other, similar institutions can assess for their automation needs.

97 MATHEMATICS AND COMPUTING↗

Stochastic Look-Ahead Commitment: A Case Study in MISO

This paper introduces the Stochastic Look Ahead Commitment (SLAC) software prototyped and tested for the Midcontinent Independent System Operator (MISO) look ahead commitment process. SLAC can incorporate hundreds of wind, load, and net scheduled interchange (NSI) uncertainty scenarios. It uses a progressive hedging method to solve a novel two-stage stochastic unit commitment. The first stage commitment decisions, made only for those generators whose decision to commit or not in each time period cannot be deferred, can cover the uncertainties within the next three hours. The second stage includes both the dispatch for each of the scenarios and the commitment decisions that can be deferred. Study results on 15 MISO production days show that SLAC may bring economic and reliability benefits under uncertainty.

MATHEMATICS AND COMPUTING↗

Google Test MPI Listener

Google Test MPI Listener is a plugin for the Google Test c++ unit testing library that organizes test output of software that uses both the MPI parallel programming model and Google Test. Typically, such output is ordered arbitrarily and disorganized, making difficult the process of interpreting test output. This plug organizes output in MPI rank order, enabling easy interpretation of test results.

Oxberry, Geoffrey↗

On-Site Inspection RadioIsotopic Spectroscopy (Osiris) System Development

We have designed and tested hardware and software for the acquisition and analysis of high-resolution gamma-ray spectra during on-site inspections under the Comprehensive Nuclear-Test-Ban Treaty (CTBT). The On-Site Inspection RadioIsotopic Spectroscopy—Osiris—software filters the spectral data to display only radioisotopic information relevant to CTBT on-site inspections, e.g.,132I. A set of over 100 fission-product spectra was employed for Osiris testing. These spectra were measured, where possible, or generated by modeling. The synthetic test spectral compositions include non-nuclear-explosion scenarios, e.g., a severe nuclear reactor accident, and nuclear-explosion scenarios such as a vented underground nuclear test. Comparing its computer-based analyses to expert visual analyses of the test spectra, Osiris correctly identifies CTBT-relevant fission product isotopes at the 95% level or better.The Osiris gamma-ray spectrometer is a mechanically-cooled, battery-powered ORTEC Transpec-100, chosen to avoid the need for liquid nitrogen during on-site inspections. The spectrometer was used successfully during the recent 2014 CTBT Integrated Field Exercise in Jordan. The spectrometer is controlled and the spectral data analyzed by a Panasonic Toughbook notebook computer. To date, software development has been the main focus of the Osiris project. In FY2016-17, we plan to modify the Osiris hardware, integrate the Osiris software and hardware, and conduct rigorous field tests to ensure that the Osiris system will function correctly during CTBT on-site inspections. The planned development will raise Osiris to technology readiness level TRL-8; transfer the Osiris technology to a commercial manufacturer, and demonstrate Osiris to potential CTBT on-site inspectors.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

ParaDyn Test Suite: Coverage Analysis

A summary and analysis of the ParaDyn source code covered by the ParaDyn Software Quality Assurance test suite is presented. The fraction of files, functions and blocks executed by the ParaDyn test suite on CPU-based architectures are 89%, 52% and 49%, respectively, as quantified by Intel’s Code Coverage Tool. When select GPU-related files are excluded, the code coverage tool shows that 89% of the files, 77% of the functions, and 56% of the code blocks are covered. It is generally considered that 70-80% coverage is a reasonable target. The primary features and general areas not currently covered by the ParaDyn test suite are identified for future inclusion in the test suite.

42 ENGINEERING↗

Introducing Human Mandible Identification [(hu)MANid]: A Free, Web‐Based GUI to Classify Human Mandibles

Abstract Statistical programs have revolutionized the way in which forensic anthropologists conduct casework by allowing practitioners to use computationally complex analytics at the click of a button. Importantly, the products of these statistical programs are reproducible and contain measures of error or uncertainty, thereby strengthening conclusions. This paper is an introduction to (hu) MAN id, a free, web‐based application that uses linear and mixture discriminant function analyses to classify human mandibles into one of many worldwide and/or periodic reference groups. The mechanics, development, and use of the application will be discussed. Further, the program was tested against other software to compare model performances and classifications. Total correct classifications among the test cases and programs were identical. Ten mandibles were tested using both statistical procedures. Mixture discriminant analysis improved classification by an average of 9.3% and correctly identified three more mandibles than LDA . Therefore, we believe (hu) MAN id will be an asset to the anthropological community.

Berg, Gregory E.↗

Verification Testing of Body Forces due to a Prescribed Angular Velocity in DYNA3D/Paradyn

This report documents the verification testing and regression testing done on a new feature in DYNA3D/Paradyn, which allows users to prescribe body force loads based upon an angular velocity. This feature is unique in that the direction of the angular velocity vector follows the unit vector formed by two coordinate points associated with two nodes or the average coordinates of two separate small collection of nodes. The angular velocity direction will follow the directional vector defined by these nodes while the angular velocity magnitude is defined by a load curve. A simple single element verification test was performed to determine the correct implementation of this feature, and two separate regression tests were added to the DYNA3D Software Quality Assur ance test suite. The nodal positions, velocities, and accelerations from the solution of the single element verification test compare well to analytically derived values of those nodal quantities. The regression tests serve as good examples of this new feature’s use case and were consequently added to the SQA test suite to ensure that further modifications of the DYNA3D source code do not unintentionally change the generated baseline answers.

42 ENGINEERING↗

Flow Sensor Test Article (F-STAr) - Assembly and Water Testing Report

The Flow Sensor Test Article (F-STAr) is a new test article under development for the Mechanisms Engineering Test Loop (METL) facility at Argonne National Laboratory. F-STAr’s purpose is to provide sodium submersible, high flowrate testing capabilities for the development of sensors, components, fluid studies and more. This report will provide a status update on the assembly and initial qualification testing of F-STAr. All components have been manufactured and received. Some modifications to these components were made to support minor changes in the test article design. Additionally, the power distribution, and data acquisition and control enclosures were designed, completed, and tested along with the control system software. Furthermore, initial qualification testing was completed in water. Finally, the F-STAr submersible Electromagnetic Flowmeter (EMFM) was completed and calibrated in flowing sodium.

42 ENGINEERING↗

Ristra Project FY23 L2 Milestone Report, Rev.1: MRT #8541: Multiphysics Scaling on EAS-3

The findings of this report were used to close out the ATDM milestone MRT# 8541, which was designed to demonstrate readiness of ATDM multiphysics codes for mission-relevant work on ATS-4, El Capitan. To this end, the closure criteria were to run a 3D shaped charge problem at scale up to 50% of the El Capitan early-access system, RZVernal (AMD Trento CPUs and AMD MI-250X GPUs), demonstrate scalability, and document challenges with the software stack and environment. LANL’s approach to this milestone was to test our modular software capability by developing an entirely new code, Moya, built upon our FleCSI framework. The physics capability and the GPU infrastructure needed for the shaped charge problem on GPUs was added to Moya, and the required calculations were performed at scale. Moya showed good scaling without any fine-tuning of GPU kernels; there is still significant room for performance enhancements, especially for the Legion backend. Tied up in this L2 milestone was a closeout of KPP-3s for the ECP ST Projects at LANL; this material will be covered in a separate document.

97 MATHEMATICS AND COMPUTING↗

Load Control System Reliability

This report summarizes the results of the Load Control System Reliability project (DOE Award DE-FC26-06NT42750). The original grant was awarded to Montana Tech April 2006. Follow-on DOE awards and expansions to the project scope occurred August 2007, January 2009, April 2011, and April 2013. In addition to the DOE monies, the project also consisted of matching funds from the states of Montana and Wyoming. Project participants included Montana Tech; the University of Wyoming; Montana State University; NorthWestern Energy, Inc., and MSE. Research focused on two areas: real-time power-system load control methodologies; and, power-system measurement-based stability-assessment operation and control tools. The majority of effort was focused on area 2. Results from the research includes: development of fundamental power-system dynamic concepts, control schemes, and signal-processing algorithms; many papers (including two prize papers) in leading journals and conferences and leadership of IEEE activities; one patent; participation in major actual-system testing in the western North American power system; prototype power-system operation and control software installed and tested at three major North American control centers; and, the incubation of a new commercial-grade operation and control software tool. Work under this grant certainly supported the DOE-OE goals in the area of “Real Time Grid Reliability Management.”

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Moment Closures on Two-Dimensional Cartesian Grids

Some moment methods for kinetic equations are complicated and take time to develop. Over the course of a couple years, this software was developed to test different closures on standard test problems in the literature. With this software, researchers in the field of moment closures will be able to rapidly test new methods.

Garrett, CharlesK.↗

Moment Closures on Two-Dimensional Cartesian Grids

Some moment methods for kinetic equations are complicated and take time to develop. Over the course of a couple years, this software was developed to test different closures on standard test problems in the literature. With this software, researchers in the field of moment closures will be able to rapidly test new methods.

Garrett, Charles K.↗

Extended Bandwidth Spread Spectrum Time Domain Reflectometry Cable Test for Thermal Aging, Low Resistance Fault, and Water Detection

In 2022, researchers at Pacific Northwest National Laboratory (PNNL) used the Accelerated and Real-Time Environmental Nodal Assessment (ARENA) cable and motor test bed to characterize spread spectrum time domain reflectometry (SSTDR) and compare the responses of an SSTDR instrument to those of a frequency domain reflectometry (FDR) instrument. Results showed both techniques could detect and locate cable anomalies such as phase-to-phase low resistance and shorts, thermal insulation damage, mechanical insulation damage, and the presence or absence of water in some conditions. The SSTDR tests used a commercial instrument provided by LiveWire Innovations Inc. This commercial instrument performed tests at 6, 12, 24, and 48 MHz bandwidth. The results of these tests were compared to FDR tests where bandwidths could be extended up to 1.3 GHz, although the best responses for cable tests were from 100 to 500 MHz. Lower bandwidth signals can propagate better along the cable while higher bandwidths have higher resolution for impedance change reflections allowing more precise indication of location and separation of anomalies. The 2022 research found that FDR responses were clearer than SSTDR and speculated that a higher bandwidth SSTDR could more successfully detect and locate cable anomalies. One advantage of the SSTDR system investigated was that it was designed for energized online use up to 1,000 volts, which may be a significant advantage for nuclear power plant use. The LiveWire SSTDR instrument is an established product in the rail and aircraft industry and updating the SSTDR hardware parameters is difficult to justify without more conclusive testing. Therefore, a software adjustable laboratory SSTDR instrument was developed by PNNL and was used to test extended bandwidth SSTDR cable tests. Within the ARENA test bed, 42 cable conditions were tested with the PNNL SSTDR, FDR, and the LiveWire SSTDR—each operating at four different bandwidths. Observations and conclusions regarding the relative performance of the three instruments over different bandwidths are note below. Responses of the PNNL SSTDR (at 50 MHz) and the LiveWire SSTDR (at 48 MHz) were similar. The PNNL SSTDR higher frequency bandwidths behaved as expected showing sharper peaks and higher noise. This validated the PNNL SSTDR as a reasonable implementation of the SSTDR technology. Lower bandwidth SSTDR responses (particularly 6 and 12 MHz) may have increased value for use within longer cables but were not particularly effective at identifying anomalous cable behavior in the 100 ft cables tested here. The higher bandwidths of the PNNL SSTDR (50, 100, 200, and 400 MHz) did not provide substantially clearer cable reflectometry responses, but having the higher frequency responses available did add to the cable test evaluation. Strong responses to shorts and low impedance faults between phases were particularly evident in the higher bandwidth PNNL SSTDR and the FDR data. Measurements were repeatable, with similar responses obtained from a thermally aged cable for tests taken a month apart. Signal noise was affected in the unshielded cable by the local in-tray cable arrangement including proximity to metal edges and rungs of the cable tray. Foam isolation of the cable from the tray metal reduced in both FDR and SSTDR responses. Cable condition monitoring in nuclear power plants will likely benefit from both more informative off-line testing methods and from the development of on-line methods for continuous monitoring of cables in use. The LWRS-funded ARENA test bed was a valuable resource for this development and direct comparison of nuclear electrical cable condition monitoring technologies. Test results are targeted to guide industry advancement of testing and monitoring tools for cable aging management.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

WISP: Watching grid Infrastructure Stealthily through Proxies (Final Technical Report)

The complex interdependencies of cyber systems (sensors and communications), physical grids and associated electricity market operations make protecting electric power grids a significant challenge. The energy sector is constantly under new, targeted, advanced and dangerous cyber-attacks that have the potential to result in the loss of human life. These threats are further exacerbated by our need to modernize the grid. One focus of cyber security research in smart grids is the securing of the SCADA system through advanced intrusion detection systems (IDS) and bad data detection algorithms in state estimation. These methods either require full knowledge of the system topology and parameters or fail to understand the physical behaviors under attack. WISP (Watching grid Infrastructure Stealthily through Proxies) is designed to provide additional protection to the power grid using only publicly available data. In particular, WISP exploits the spatio-temporal nature of the real time locational marginal prices (LMPs), in conjunction with other information such as bids, weather, outages and load data to analyze anomalous power pricing behaviors and then correlate those observations to localize regions of interest and identify potential cyber events. WISP is non-intrusive as the tool is deployed as a service in the Cloud or on premise and provides reliable information to system operators for enhanced situational awareness, without impeding energy delivery functions. The WISP technology comprises three modules: the data-driven anomaly detection core, the vulnerability and risk analysis and the root cause analysis. The data-driven anomaly detection core performs the tasks of feature selection, anomaly detection and attack region localization. The vulnerability and risk analysis module provides system level information of the vulnerable variables and times, assisting the operators in selecting monitoring and protection nodes. The root cause analysis module takes the detection results and identifies potential operational conditions that contribute to the detected anomalies. In Phase I, we have demonstrated the feasibility and effectiveness of WISP. We developed a realistic electricity market simulator capable of generating normal and attack market data under various operational conditions. We developed a series of cyber-attack detection and analysis algorithms and evaluated them under multiple data sources. Finally, we integrated all modules into an end-to-end software, providing functions for data management, data analytics and visualization. Specifically, we have achieved: (i) real-time data acceptance from external utility interfaces with >99% acceptance rate; (ii) high performance anomaly detection algorithms with >98% detection accuracy and <0.1% false alarm rate; and (iii) ultra-low computing delay <50 milliseconds. Additionally, our team developed algorithms to identify the vulnerable variables in electricity market operations and root cause analysis functions to identify major contributors to the price spikes. These ancillary modules are necessary when deploying WISP in real world industry environment. In Phase II, we have demonstrated the effectiveness of WISP software on realistic largescale power systems. We performed red team testing for the Phase I WISP software and identified software vulnerabilities and implemented corresponding mitigation solutions. We adapted the electricity market simulator for the Texas synthetic 2000-bus system and generated datasets for the false data injection attacks. We created database and visualization interfaces for the Texas system and the ISO New England system. We performed software optimization in terms of operation efficiency, computing speed and detection accuracy. Finally, we tested the software on the Texas system and the ISO New England system and evaluated the detection performance. Overall, we achieved above 89% detection rate, below 3% false alarm rate and below 37 seconds of end-to-end detection delay.

24 POWER TRANSMISSION AND DISTRIBUTION↗