Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Making or Breaking a Rover: System Engineering Parameters On-Board the Mars 2020 Perseverance Rover

On February 18, 2021, Perseverance, NASA’s Jet Propulsion Laboratory’s (JPL’s) Mars 2020 Rover, successfully landed on Mars with all systems nominal, despite the risk surrounding the over 200,000 internal flight parameters that had to be properly configured. The Perseverance team defines these parameters as software variables that are configurable, commandable and retrievable from Earth. In 2015, the Mars 2020 project leaders focused on improving systems engineering of parameters based on their experiences from parameter management on previous Mars rovers (Curiosity, Opportunity, Spirit, and Pathfinder) and parameter failures of past missions, such as the mission-ending parameter of the Mars Climate Orbiter. The new rigorous development process allowed for efficient certification and effective implementation of the parameters, allowing the rover to approach and land on the red planet (the most challenging phase of the mission) with zero parameter issues. Although successful, the Perseverance team learned many lessons for how to better manage parameters for the continued surface operations of the Mars 2020 mission and future missions. This paper will discuss eight parameter-management topics for the Perseverance Mission. The first is parameter definition: how we define parameters on our mission, where they are physically located on the vehicle, and why we have so many of them. The second topic is the updated parameter flight software module from Curiosity, including details on the 99% reduction in parameter commands, new bulk configuration capabilities, and improved parameter traceability. The third topic is parameter selection for different mission phases; this includes improving and tweaking our preferred parameter settings until they become certification candidates and managing parameter configurations based on test venue throughout the mission life cycle. The fourth topic is our flight certification process; this includes certification of flight values for four different epochs in the mission: Launch, Entry Decent and Landing (EDL) - 6days, Landing + 5 Sols (Martian Days, still on Cruise Flight Software), and once are on Surface Flight Software (FSW). The fifth topic covers in-flight command implementation, along with details on testing, validation, and verification of those commands. In the sixth section, we will explain our use of open-source management tools, including how we used GitHub for version control and management approvals. The seventh topic will describe the ground tools used in operations, including capabilities of the in-house built tool called Parasol. The eighth and final topic will dig into lessons learned for improving parameter management in the future of this mission and others.

Roth, Brian↗
NASA NTRS2022

Making or Breaking a Rover- Systems Engineering Parameters On-Board the Mars 2020 Perseverance Rover

On February 18, 2021, Perseverance, NASA’s Jet Propulsion Laboratory’s (JPL’s) Mars 2020 Rover, successfully landed on Mars with all systems nominal, despite the risk surrounding the over 200,000 internal flight parameters that had to be properly configured. The Perseverance team defines these parameters as software variables that are configurable, commandable and retrievable from Earth. In 2015, the Mars 2020 project leaders focused on improving systems engineering of parameters based on their experiences from parameter management on previous Mars rovers (Curiosity, Opportunity, Spirit, and Pathfinder) and parameter failures of past missions, such as the mission-ending parameter of the Mars Climate Orbiter. The new rigorous development process allowed for efficient certification and effective implementation of the parameters, allowing the rover to approach and land on the red planet (the most challenging phase of the mission) with zero parameter issues. Although successful, the Perseverance team learned many lessons for how to better manage parameters for the continued surface operations of the Mars 2020 mission and future missions. This paper will discuss eight parameter-management topics for the Perseverance Mission. The first is parameter definition: how we define parameters on our mission, where they are physically located on the vehicle, and why we have so many of them. The second topic is the updated parameter flight software module from Curiosity, including details on the 99% reduction in parameter commands, new bulk configuration capabilities, and improved parameter traceability. The third topic is parameter selection for different mission phases; this includes improving and tweaking our preferred parameter settings until they become certification candidates and managing parameter configurations based on test venue throughout the mission life cycle. The fourth topic is our flight certification process; this includes certification of flight values for four different epochs in the mission: Launch, Entry Decent and Landing (EDL) - 6days, Landing + 5 Sols (Martian Days, still on Cruise Flight Software), and once are on Surface Flight Software (FSW). The fifth topic covers in-flight command implementation, along with details on testing, validation, and verification of those commands. In the sixth section, we will explain our use of open-source management tools, including how we used GitHub for version control and management approvals. The seventh topic will describe the ground tools used in operations, including capabilities of the in-house built tool called Parasol. The eighth and final topic will dig into lessons learned for improving parameter management in the future of this mission and others.

Roth, Brian↗
DOE OSTI2022

Modelica-json: Transforming energy models to digitize the control delivery process

Building simulation models are typically not used to generate the documentation required for bidding and project delivery of commercial building systems, or for their semantic modeling and commissioning. This paper presents a software tool that aids in digitizing the control delivery process, spanning simulation during design to implementation and formal verification during commissioning. The tool can generate from Modelica models digital documentation of control sequences. This digital documentation, along with other project drawings and specifications can be used for project bidding. It can also be used for implementation of control sequences through machineto-machine translation to commercial legacy control products, for which we are currently developing the proposed ASHRAE Standard 231P based on the presented work. Moreover, as-installed control sequences can be formally verified against the design specification, and a semantic model in Brick can be exported to aid in configuration of building analytics and fault detection. The paper presents what we believe is the first translation of a Modelica-implemented control sequence to a native implementation on a commercial control platform, using the webCTRL product line from Automated Logic. The paper also shows how a webCTRL implementation can be formally verified against its Modelica specification. These use cases have all been demonstrated with a prototype implementation that is now being further developed.

Wetter, Michael↗
DOE OSTI2014

Absolute calibration of the Gamma Knife{sup ®} Perfexion™ and delivered dose verification using EPR/alanine dosimetry

Purpose: Elekta Leksell Gamma Knife{sup ®} (LGK) is a radiotherapy beam machine whose features are not compliant with the international calibration protocols for radiotherapy. In this scope, the Laboratoire National Henri Becquerel and the Pitié-Salpêtrière Hospital decided to conceive a new LKG dose calibration method and to compare it with the currently used one. Furthermore, the accuracy of the dose delivered by the LGK machine was checked using an “end-to-end” test. This study also aims to compare doses delivered by the two latest software versions of the Gammaplan treatment planning system (TPS). Methods: The dosimetric method chosen is the electron paramagnetic resonance (EPR) of alanine. Dose rate (calibration) verification was done without TPS using a spherical phantom. Absolute calibration was done with factors calculated by Monte Carlo simulation (MCNP-X). For “end-to-end” test, irradiations in an anthropomorphic head phantom, close to real treatment conditions, are done using the TPS in order to verify the delivered dose. Results: The comparison of the currently used calibration method with the new one revealed a deviation of +0.8% between the dose rates measured by ion chamber and EPR/alanine. For simple fields configuration (less than 16 mm diameter), the “end-to-end” tests showed out average deviations of −1.7% and −0.9% between the measured dose and the calculated dose by Gammaplan v9 and v10, respectively. Conclusions: This paper shows there is a good agreement between the new calibration method and the currently used one. There is also a good agreement between the calculated and delivered doses especially for Gammaplan v10.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2023

Requirements Description of EvaluateFlux

This report presents the modeling and simulation capabilities of Argonne National Laboratory’s EvaluateFlux code [1] that is used in present reactor analysis activities. These capabilities will then be used to establish the set of EvaluateFlux verification tasks necessary to verify EvaluateFlux for usage on commercial projects. A similar approach was taken for the PERSENT, REBUS and DIF3D software packages. The EvaluateFlux program is a post-processing utility program for DIF3D, specifically DIF3D-VARIANT. As covered in [1], the EvaluateFlux program was built to allow users to obtain flux and power traverses through the domain. Its primary purpose was to facilitate foil analysis by evaluating the flux solution from DIF3D-VARIANT and combining it with foil cross section data. The outputs from EvaluateFlux are contained in four possible output files: FluxAndRegionRR.out, IsotopeMicroRR.out, LabeledRegionRR.out, and IsotopeMacroRR.out. Each file is focused on using the flux evaluate and cross section data in a different manner and all are covered in this verification process.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2014

NEAMS Software V&V Plan for the MARMOT Software

In order to ensure the accuracy and quality of the microstructure based materials models being developed in conjunction with MARMOT simulations, MARMOT must undergo exhaustive verification and validation. Only after this process can we confidently rely on the MARMOT code to predict the microstructure evolution within the fuel. Therefore, in this report we lay out a V&V plan for the MARMOT code, highlighting where existing data could be used and where new data is required.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

Verification Testing For Solid-Element Material Models 11-19 in DYNA3D/ParaDyn

This technical report documents the creation and implementation of verification tests for solid-element material models 11 through 19 available in DYNA3D/ParaDyn. The verification tests covered all aspects of each material model, except for the Weibull distribution functionality in material models 15 and 19. General test cases were created to verify the elastic and plastic behavior of the material models. Other additional tests were developed to examine the intricacies of each material model. Each test involved the use a kinematic load case and specification of material parameters necessary to activate corresponding features of the material model. The load cases prescribed the full time history of the kinematic motion for the solid elements, and these loads are independent of the material model or element formulation. When possible, closed form solutions were then derived for each verification test in a continuum setting. The DYNA3D simulations for each test were carried out over a unit time interval, t ϵ [0, 1], and the as implemented DYNA3D response was compared to the closed form solutions evaluated at discrete points in time. A relative error measure was determined for each test to justify the proper implementation of the material model. The relative errors comparing the DYNA3D solution to the analytical solution were, in general, on the order of machine precision except where noted. This signifies the proper implementation of solid-element material models 11-19. In the development of these verification tests, six bugs were found and fixed in the source code. Additionally, this work generated eighteen DYNA3D input decks and answer extraction scripts in the DYNA3D/ParaDyn Software Quality Assurance test suite, which are comprised of a total of 285 solid-element tests. Testing for each material model utilizes two input decks and answer extraction scripts, where one focuses on the linear elastic response and the other examines the inelastic and remaining functionalities of the material model. In total, this work added 285 individual verification test problems in the DYNA3D/ParaDyn test suite.

42 ENGINEERING↗
NASA NTRS2009

Oxygen Generation System Laptop Bus Controller Flight Software

The Oxygen Generation System Laptop Bus Controller Flight Software was developed to allow the International Space Station (ISS) program to activate specific components of the Oxygen Generation System (OGS) to perform a checkout of key hardware operation in a microgravity environment, as well as to perform preventative maintenance operations of system valves during a long period of what would otherwise be hardware dormancy. The software provides direct connectivity to the OGS Firmware Controller with pre-programmed tasks operated by on-orbit astronauts to exercise OGS valves and motors. The software is used to manipulate the pump, separator, and valves to alleviate the concerns of hardware problems due to long-term inactivity and to allow for operational verification of microgravity-sensitive components early enough so that, if problems are found, they can be addressed before the hardware is required for operation on-orbit. The decision was made to use existing on-orbit IBM ThinkPad A31p laptops and MIL-STD-1553B interface cards as the hardware configuration. The software at the time of this reporting was developed and tested for use under the Windows 2000 Professional operating system to ensure compatibility with the existing on-orbit computer systems.

Rowe, Chad↗
NASA NTRS2005

Thermal System Verification and Model Validation for NASA's Cryogenic Passively Cooled James Webb Space Telescope

A thorough and unique thermal verification and model validation plan has been developed for NASA s James Webb Space Telescope. The JWST observatory consists of a large deployed aperture optical telescope passively cooled to below 50 Kelvin along with a suite of several instruments passively and actively cooled to below 37 Kelvin and 7 Kelvin, respectively. Passive cooling to these extremely low temperatures is made feasible by the use of a large deployed high efficiency sunshield and an orbit location at the L2 Lagrange point. Another enabling feature is the scale or size of the observatory that allows for large radiator sizes that are compatible with the expected power dissipation of the instruments and large format Mercury Cadmium Telluride (HgCdTe) detector arrays. This passive cooling concept is simple, reliable, and mission enabling when compared to the alternatives of mechanical coolers and stored cryogens. However, these same large scale observatory features, which make passive cooling viable, also prevent the typical flight configuration fully-deployed thermal balance test that is the keystone to most space missions thermal verification plan. JWST is simply too large in its deployed configuration to be properly thermal balance tested in the facilities that currently exist. This reality, when combined with a mission thermal concept with little to no flight heritage, has necessitated the need for a unique and alternative approach to thermal system verification and model validation. This paper describes the thermal verification and model validation plan that has been developed for JWST. The plan relies on judicious use of cryogenic and thermal design margin, a completely independent thermal modeling cross check utilizing different analysis teams and software packages, and finally, a comprehensive set of thermal tests that occur at different levels of JWST assembly. After a brief description of the JWST mission and thermal architecture, a detailed description of the three aspects of the thermal verification and model validation plan is presented.

Cleveland, Paul E.↗
DOE OSTI2019

OSIRIS Validation and Verification Report

The On-Site Inspection RadioIsotopic Spectroscopy (OSIRIS) system is a gamma-ray spectroscopy instrument for on-site inspections conducted under the Comprehensive Nuclear-Test-Ban Treaty (CTBT). The OSIRIS software filters spectral analysis data, revealing information to users on only 17 CTBT-relevant fission-product isotopes. In this report, we review the CTBT requirements for on-site inspection radionuclide identifiers, plus additional requirements imposed by the design team. Then a series of validation tests conducted during OSIRIS development are described. The validation tests were designed to gauge if the nascent OSIRIS system met those requirements. We conclude with a set of recommended verification tests to ensure that newly-manufactured OSIRIS systems also will meet the requirements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2020

Is Structured Agile an Oxymoron? Tales from Implementing and Executing Agile in a US Government Environment

To paraphrase a famous quote, "No plan survives contact with the reality." Software (SW) development is often a classic example of this: whatever the plan was for a particular development, it often does not survive contact with technical realities, budget realities, program realities and schedule realities. Traditionally, SW development has followed a waterfall methodology with requirements being rigorously specified before the design, which was completed before the coding and unit testing started, which were in turn finished before validation and verification started. This model of SW engineering derives much from the HW engineering of large systems, and has been the standard methodology used in US government software acquisitions and systems for decades, with highly variable results. US Government SW requirements are built around Waterfall concepts, which assume that the plan will survive contact with reality, or at least that modifications to the plan are relatively small, and relatively few.Because of the inefficiencies and difficulties inherent in Waterfall, the commercial SW world started using a different SW development methodology called Agile more than 20 years ago. Agile believes that a plan should evolve and learn rapidly in response to the realities encountered. At its core, there are a few key elements of Agile:- A small team of people which is highly flexible and adaptive. The team collaborates and interoperates through sophisticated development architectures and release environments- An iterative, incremental development and release approach which is based upon the concept that knowledge comes from experience within the team, and that the team makes decisions based upon what it knows- A team culture which prizes transparency, inspection and adaptation. These values are necessary so that the team experience and decision making is transparent and responsive to the realities encountered during development and testingSo, how to use Agile in a US Government environment? GMSEC (Goddard Mission Services Evolution Center) develops satellite ground system software for NASA and other US Government agencies. The SW developed by the team contains a large code base of many applications used within satellite mission operations centers. It spans the full gamut of SW development types: from SW which is in a classic maintenance and sustainment mode, to new developments with a fairly well understood scope and approach, to new developments whose scope and approach are quite unclear and which require significant research and prototyping. Team members move between all of these different types of SW development. Waterfall was inadequate to the programmatic and technical needs of the team, as well as the various types of SW development being done. The software plan was not surviving contact with the technical and programmatic realities experienced by the team. To address this, the team started a small pilot project in 2016 to test the use of Agile within a small subset of the team for a new web services application. In early 2018, the use of Agile was expanded to the whole team and all the software, but we had to fulfill the NASA SW development requirements. And we needed to do this while still remaining true to the key Agile elements of transparency, inspection and adaption. In order to do this, the team worked very closely with the Software Process Improvement (SPI) team at NASA Goddard, as well as NASA engineering manageme

Beech, Theresa W.↗
NASA NTRS1996

Software Development Technologies for Reactive, Real-Time, and Hybrid Systems

The research is directed towards the design and implementation of a comprehensive deductive environment for the development of high-assurance systems, especially reactive (concurrent, real-time, and hybrid) systems. Reactive systems maintain an ongoing interaction with their environment, and are among the most difficult to design and verify. The project aims to provide engineers with a wide variety of tools within a single, general, formal framework in which the tools will be most effective. The entire development process is considered, including the construction, transformation, validation, verification, debugging, and maintenance of computer systems. The goal is to automate the process as much as possible and reduce the errors that pervade hardware and software development.

Manna, Zohar↗
DOE OSTI2016

DRSPALL: Impact of the Modification of the Numerical Spallings Model on Waste Isolation Pilot Plant Performance Assessment.

The numerical code DRSPALL (from direct release spallings) is written to calculate the volume of Waste Isolation Pilot Plant (WIPP) solid waste subject to material failure and transport to the surface as a result of a hypothetical future inadvertent drilling intrusion. An error in the implementation of the DRSPALL finite difference equations was discovered as documented in Software Problem Report (SPR) 13-001. The modifications to DRSPALL to correct the finite difference equations are detailed, and verification and validation testing has been completed for the modified DRSPALL code. The complementary cumulative distribution function (CCDF) of spallings releases obtained using the modified DRSPALL is higher compared to that found in previous WIPP performance assessment (PA) calculations. Compared to previous PAs, there was an increase in the number of vectors that result in a nonzero spallings volume, which generally translates to an increase in spallings releases. The overall mean CCDFs for total releases using the modified DRSPALL are virtually unchanged, thus the modification to DRSPALL did not impact WIPP PA calculation results.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Implementation of Plot File Testing in the DYNA3D/ParaDyn Software Quality Assurance Suite

Automated testing of DYNA3D/ParaDyn plot files was added to the DYNA3D/ParaDyn software quality assurance (SQA) test suite. The new capability extracts select data from the plot files generated during each verification run and compares it to the same baseline answers used to verify the problem. Deviations between baseline answers and plot file values are reported in the same manner as solution discrepancies, and differences in precision levels between the baseline answers and plot file results are accounted for. The new testing leverages the existing SQA test suite framework and test problems and the Python Mili reader and minimally increases the overall run time (< 5%) of the SQA test suite. This new capability provides incremental end-toend testing of the most common DYNA3D/ParaDyn simulation workflows.

42 ENGINEERING↗
DOE OSTI2019

Regression for Assessing Contaminant Plume Trends

The Department of Energy’s (DOE) Hanford site has significant dissolved-phase groundwater contaminant plumes, for which remediation activities are ongoing. Assessing concentration trends is critical for making decisions regarding remedy implementation or performance. The SOCRATES single-page web application provides a quality-assured framework for data access and consistent data analytics using standard methods. To provide trend information for remedial decisions, statistical analyses based on ordinary least squares (OLS) are already supported by SOCRATES. The presence of censored data, however, can skew the result of an OLS regression. Other methods, such as the Akritas-Theil-Sen (ATS) or Tobit regressions can be more powerful. These two regression methods are used by Hanford site personnel to analyze groundwater contaminant concentration data, which often include nondetects (censored data). The ATS method uses comparisons of observation-pair ranks and the Tobit method uses a maximum likelihood estimation method. The Tobit method can also accommodate covariates, such as river stage or groundwater level fluctuations. Both ATS and Tobit regressions can be performed with the statistics software R, but to incorporate these methods into SOCRATES, their R code was ported into JavaScript. For quality assurance verification, the new code was tested with hexavalent chromium concentration data from the Hanford 100-K West Area and compared to regression analyses that were previously conducted on this data using R.

100-K Area↗
DOE OSTI2026

Testing the Common Platform LLRF with a 197 MHz NCRF Cavity

The Common Platform is the hardware that will support the new LLRF platform to be used for the Electron-Ion Collider. The Common Platform features a carrier board that is used to interface with a variety of daughter for different applications. This paper details the testing that was done using the Common Platform and an RF Digitizer Daugther Board. Firstly, the firmware and software development is discussed followed by a description of the controls algorithms used in the testing. Then, testing and verification of the platform with a 197 MHz NCRF cavity is discussed. The paper concludes with results from the testing and the path forward.

42 ENGINEERING↗
NASA NTRS1992

Expert system verification and validation study: ES V/V Workshop

The primary purpose of this document is to build a foundation for applying principles of verification and validation (V&V) of expert systems. To achieve this, some V&V as applied to conventionally implemented software is required. Part one will discuss the background of V&V from the perspective of (1) what is V&V of software and (2) V&V's role in developing software. Part one will also overview some common analysis techniques that are applied when performing V&V of software. All of these materials will be presented based on the assumption that the reader has little or no background in V&V or in developing procedural software. The primary purpose of part two is to explain the major techniques that have been developed for V&V of expert systems.

French, Scott↗