Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software quality, testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
NASA NTRS1988

Application of intelligent systems to wind tunnel test facilities

An approach to the application of intelligent-systems technology to the wind tunnel facilities at NASA Ames Research Center is outlined. To help fulfill the long-range goals of improving data quality and increasing personnel efficiency and management effectiveness, three major areas of intelligent systems application are recommended. The available state-of-the-art technology for developing the proposed systems is reviewed including the application of commercial software packages. The initial tasks and effort to develop these systems are recommended. A prototype expert system for selection of internal strain-gage balances has been built and is presented herein as an example model for the future systems.

Lo, Ching F.↗
DOE OSTI2025

MaPSA Quality Control and AI-Enhanced Grading For the CMS Phase-II Tracker Upgrade

The Compact Muon Solenoid (CMS) experiment will undergo changes as part of the Large Hadron Collider upgrade. The CMS tracker will be upgraded to cope with the new radiation environment and to provide tracking at the first level trigger. This upgrade features a new type of silicon module called PS Module, which combines a Pixel sensor and a Strip sensor in the same module. The pixel portion of the PS module has a sensor bump bonded to 16 Macro Pixel ASICs (MPA) to form a Macro Pixel Sub Assembly (MaPSA). At Fermilab, MaPSAs are tested for quality control before being assembled with the strip sensors, readout and service electronics to form a PS Module. All of this test data is stored in a centralized database, and is used to grade the final module to determine if it will be installed in the detector. The Phase II Outer Tracker Analyzer of Test Outputs (POTATO) is the software that processes this data and determines the module grades. Using recent technologies, an AI agent is being im plemented into POTATO in order to allow users to more efficiently sort through the large amounts of analysis data and ensure that only the user specified data is being considered. This poster will display the process of testing a MaPSA, how that test data is relevant to module assembly and grading, and how the POTATO grading tool is being improved with the use of an embedded AI agent.

Gzamouranis, Olivia [Purdue U.]↗
NASA NTRS2017

Aeroacoustic Simulations of a Nose Landing Gear with FUN3D: A Grid Refinement Study

A systematic grid refinement study is presented for numerical simulations of a partially-dressed, cavity-closed (PDCC) nose landing gear configuration that was tested in the University of Florida's open-jet acoustic facility known as the UFAFF. The unstructured-grid flow solver FUN3D is used to compute the unsteady flow field for this configuration. Mixed-element grids generated using the Pointwise (Registered Trademark) grid generation software are used for numerical simulations. Particular care is taken to ensure quality cells and proper resolution in critical areas of interest in an effort to minimize errors introduced by numerical artifacts. A set of grids was generated in this manner to create a family of uniformly refined grids. The finest grid was then modified to coarsen the wall-normal spacing to create a grid suitable for the wall-function implementation in FUN3D code. A hybrid Reynolds-averaged Navier-Stokes/large eddy simulation (RANS/LES) turbulence modeling approach is used for these simulations. Time-averaged and instantaneous solutions obtained on these grids are compared with the measured data. These CFD solutions are used as input to a FfowcsWilliams-Hawkings (FW-H) noise propagation code to compute the farfield noise levels. The agreement of the computed results with the experimental data improves as the grid is refined.

Vatsa, Veer N.↗
NASA NTRSDate not supplied

Service Provider Agnostic Adaptive Coding and Modulation System Design

Commercial service providers (CSPs) are increasingly used to provide communications services to near-Earth science and exploration missions. CSPs commonly utilize modems that are designed to implement adaptive coding and modulation (ACM) in unique ways with modems from the same vendor. However, some missions cannot fly a matching vendor modem, either because the vendor only designs modems for terrestrial use, or the mission needs to fly a software defined radio. We present an ACM system design that allows missions to implement ACM on DVB-S2 compliant CSP links without needing to fly a modem from the same vendor the CSP uses. We implement and test our design in an emulated communications testbed. These tests demonstrate that our ACM system design can successfully adapt to link quality changes up to 6.12 dB/second.

Adaptive coding and modulation↗
NASA NTRSDate not supplied

Service Provider Agnostic Adaptive Coding and Modulation System Design

Commercial service providers (CSPs) are increasingly used to provide communications services to near-Earth science and exploration missions. CSPs commonly utilize modems that are designed to implement adaptive coding and modulation (ACM) in unique ways with modems from the same vendor. However, some missions cannot fly a matching vendor modem, either because the vendor only designs modems for terrestrial use, or the mission needs to fly a software defined radio. We present an ACM system design that allows missions to implement ACM on DVB-S2 compliant CSP links without needing to fly a modem from the same vendor the CSP uses. We implement and test our design in an emulated communications testbed. These tests demonstrate that our ACM system design can successfully adapt to link quality changes up to 6.12 dB/second.

Adaptive coding and modulation↗
DOE OSTI2020

A microchip microcontroller-based transducer controller for non-contact scanning probe microscopy with phase-locked loop, amplitude, and Q control

An inexpensive yet versatile transducer controller for non-contact scanning probe microscopy (SPM) based on a PIC32 microcontroller from Microchip Technology, Inc is described. In addition to feedback control using the amplitude or phase of the signal from the non-contact transducer, the controller includes a phase-locked loop for frequency-shift feedback, as well as fixed-amplitude, quality factor (Q) control, and self-excitation modes. Apart from the input amplifiers, output buffers, and the Q-control circuit, all other functions of the controller are instantiated in software on the microchip, enabling rapid changes in operating parameters if needed. The controller communicates with a host personal computer via a simple serial connection. The controller has been tested with a quartz tuning-fork transducer but can be used with any oscillating non-contact transducer.

47 OTHER INSTRUMENTATION↗
NASA NTRS1989

NASA-LaRc Flight-Critical Digital Systems Technology Workshop

The outcome is documented of a Flight-Critical Digital Systems Technology Workshop held at NASA-Langley December 13 to 15 1988. The purpose of the workshop was to elicit the aerospace industry's view of the issues which must be addressed for the practical realization of flight-critical digital systems. The workshop was divided into three parts: an overview session; three half-day meetings of seven working groups addressing aeronautical and space requirements, system design for validation, failure modes, system modeling, reliable software, and flight test; and a half-day summary of the research issues presented by the working group chairmen. Issues that generated the most consensus across the workshop were: (1) the lack of effective design and validation methods with support tools to enable engineering of highly-integrated, flight-critical digital systems, and (2) the lack of high quality laboratory and field data on system failures especially due to electromagnetic environment (EME).

Meissner, C. W., Jr.↗
NASA NTRS2001

Verifying Data Integrity of Electronically Scanned Pressure Systems at the NASA Glenn Research Center

The proper operation of the Electronically Scanned Pressure (ESP) System critical to accomplish the following goals: acquisition of highly accurate pressure data for the development of aerospace and commercial aviation systems and continuous confirmation of data quality to avoid costly, unplanned, repeat wind tunnel or turbine testing. Standard automated setup and checkout routines are necessary to accomplish these goals. Data verification and integrity checks occur at three distinct stages, pretest pressure tubing and system checkouts, daily system validation and in-test confirmation of critical system parameters. This paper will give an overview of the existing hardware, software and methods used to validate data integrity.

Panek, Joseph W.↗
DOE OSTI2024

Automating the Analysis of Large Language Models Responses through Zero-Shot Question Answering

Recent advancements in Large Language Models (LLMs) have shown significant potential in various applications, yet their evaluation, particularly in zero-shot question answering scenarios, remains a challenging task. In this study, our objective was to explore precision metrics for Large Language Models (LLM) and design and implement a software pipeline to automatically evaluate LLMs' outputs under zero-shot question answering. Zero-shot question answering involves a model providing answers to questions about topics it hasn't seen during training. It leverages the principles of zero-shot learning by relying on semantic understanding and generalization from related knowledge. The data used was metadata from medical databases on congenital heart disease. We explored eleven LLM metrics and selected three for our evaluation: BLEU, BERTScore, and MoverScore. BLEU calculates a score based on the overlap of n-grams (contiguous sequences of n items, typically words) between the machine-generated translation and the reference translations. Higher BLEU scores indicate better correspondence between the machine-generated and human-generated translations. BERTScore is a metric used to evaluate the quality of machine-generated text by measuring the similarity of token embeddings produced by BERT (Bidirectional Encoder Representations from Transformers) between the generated text and reference text. MoverScore is a metric that quantifies the dissimilarity between the distributions of word embeddings from machine-generated text and reference text, emphasizing semantic similarity over exact token overlap. We also introduced HBKI, a composite metric summarizing these approaches. We tested five models —GPT-3, Llama-2, Gemini 1.5 Pro, Solar 10.7B, and Mixtral-8x7b. Our software pipeline, designed and implemented using Object-Oriented Programming principles, allows users to customize the selection and extraction of features for topics of interest in their own research. Our results show that MoverScore delivered the most precise evaluation of the LLM's outputs, while Mixtral-8x7b achieved the best overall performance in extracting metadata from the databases.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

Space shuttle main engine anomaly data and inductive knowledge based systems: Automated corporate expertise

Progress is reported on the development of SCOTTY, an expert knowledge-based system to automate the analysis procedure following test firings of the Space Shuttle Main Engine (SSME). The integration of a large-scale relational data base system, a computer graphics interface for experts and end-user engineers, potential extension of the system to flight engines, application of the system for training of newly-hired engineers, technology transfer to other engines, and the essential qualities of good software engineering practices for building expert knowledge-based systems are among the topics discussed.

Modesitt, Kenneth L.↗
NASA NTRS2007

Second-Generation Electronic Nose

A report discusses the second generation of the JPL Electronic Nose (ENose), an array of 32 semi-specific chemical sensors used as an event monitor to identify and quantify contaminants released into breathing air by leaks or spills. It is designed to monitor the environment for changes in air quality, and is trained to identify and quantify selected chemical species at predetermined concentrations, ranging from sub-ppm to ppth. This system has improved reproducibility for making matched arrays, allowing use of data analysis software with minimal recalibration on sensor set replacement. The Second Generation (SG) ENose is a follow-up to the first JPL Electronic Nose that was tested on an earlier space shuttle mission (STS-95). Improvements have been made to the hardware, sensor materials, and data analysis software.

Homer, Margie↗
NASA NTRS2008

Assessing Requirements Quality through Requirements Coverage

In model-based development, the development effort is centered around a formal description of the proposed software system the model. This model is derived from some high-level requirements describing the expected behavior of the software. For validation and verification purposes, this model can then be subjected to various types of analysis, for example, completeness and consistency analysis [6], model checking [3], theorem proving [1], and test-case generation [4, 7]. This development paradigm is making rapid inroads in certain industries, e.g., automotive, avionics, space applications, and medical technology. This shift towards model-based development naturally leads to changes in the verification and validation (V&V) process. The model validation problem determining that the model accurately captures the customer's high-level requirements has received little attention and the sufficiency of the validation activities has been largely determined through ad-hoc methods. Since the model serves as the central artifact, its correctness with respect to the users needs is absolutely crucial. In our investigation, we attempt to answer the following two questions with respect to validation (1) Are the requirements sufficiently defined for the system? and (2) How well does the model implement the behaviors specified by the requirements? The second question can be addressed using formal verification. Nevertheless, the size and complexity of many industrial systems make formal verification infeasible even if we have a formal model and formalized requirements. Thus, presently, there is no objective way of answering these two questions. To this end, we propose an approach based on testing that, when given a set of formal requirements, explores the relationship between requirements-based structural test-adequacy coverage and model-based structural test-adequacy coverage. The proposed technique uses requirements coverage metrics defined in [9] on formal high-level software requirements and existing model coverage metrics such as the Modified Condition and Decision Coverage (MC/DC) used when testing highly critical software in the avionics industry [8]. Our work is related to Chockler et al. [2], but we base our work on traditional testing techniques as opposed to verification techniques.

Rajan, Ajitha↗
NASA NTRS2007

Compact Microwave Mercury Ion Clock for Space Applications

We review progress in developing a small Hg ion clock for space operation based on breadboard ion-clock physics package where Hg ions are shuttled between a quadrupole and a 16-pole rf trap. With this architecture we have demonstrated short-term stability approx.1-2x10(exp -13) at 1 second, averaging to 10-15 at 1 day. This development shows that H-maser quality stabilities can be produced in a small clock package, comparable in size to an ultra-stable quartz oscillator required or holding 1-2x10(exp -13) at 1 second. We have completed an ion clock physics package designed to withstand vibration of launch and are currently building a approx. 1 kg engineering model for test. We also discuss frequency steering software algorithms that simultaneously measure ion signal size and lamp light output, useful for long term operation and self-optimization of microwave power and return engineering data.

navigation↗
NASA NTRS2015

Design of a Thermal Precipitator for the Characterization of Smoke Particles from Common Spacecraft Materials

A thermal precipitator (TP) was designed to collect smoke aerosol particles for microscopic analysis in fire characterization research. Information on particle morphology, size and agglomerate structure obtained from these tests supplements additional aerosol data collected. Modeling of the thermal precipitator throughout the design process was performed with the COMSOL Multiphysics finite element software package, including the Eulerian flow field and thermal gradients in the fluid. The COMSOL Particle Tracing Module was subsequently used to determine particle deposition. Modeling provided optimized design parameters such as geometry, flow rate and temperatures. The thermal precipitator was built and testing verified the performance of the first iteration of the device. The thermal precipitator was successfully operated and provided quality particle samples for microscopic analysis, which furthered the body of knowledge on smoke particulates. This information is a key element of smoke characterization and will be useful for future spacecraft fire detection research.

Smoke↗
NASA NTRS2009

Extreme Programming: Maestro Style

"Extreme Programming: Maestro Style" is the name of a computer programming methodology that has evolved as a custom version of a methodology, called extreme programming that has been practiced in the software industry since the late 1990s. The name of this version reflects its origin in the work of the Maestro team at NASA's Jet Propulsion Laboratory that develops software for Mars exploration missions. Extreme programming is oriented toward agile development of software resting on values of simplicity, communication, testing, and aggressiveness. Extreme programming involves use of methods of rapidly building and disseminating institutional knowledge among members of a computer-programming team to give all the members a shared view that matches the view of the customers for whom the software system is to be developed. Extreme programming includes frequent planning by programmers in collaboration with customers, continually examining and rewriting code in striving for the simplest workable software designs, a system metaphor (basically, an abstraction of the system that provides easy-to-remember software-naming conventions and insight into the architecture of the system), programmers working in pairs, adherence to a set of coding standards, collaboration of customers and programmers, frequent verbal communication, frequent releases of software in small increments of development, repeated testing of the developmental software by both programmers and customers, and continuous interaction between the team and the customers. The environment in which the Maestro team works requires the team to quickly adapt to changing needs of its customers. In addition, the team cannot afford to accept unnecessary development risk. Extreme programming enables the Maestro team to remain agile and provide high-quality software and service to its customers. However, several factors in the Maestro environment have made it necessary to modify some of the conventional extreme-programming practices. The single most influential of these factors is that continuous interaction between customers and programmers is not feasible.

Norris, Jeffrey↗
NASA NTRS2011

Safety and Mission Assurance for In-House Design Lessons Learned from Ares I Upper Stage

This viewgraph presentation identifies lessons learned in the course of the Ares I Upper Stage design and in-house development effort. The contents include: 1) Constellation Organization; 2) Upper Stage Organization; 3) Presentation Structure; 4) Lesson-Importance of Systems Engineering/Integration; 5) Lesson-Importance of Early S&MA Involvement; 6) Lesson-Importance of Appropriate Staffing Levels; 7) Lesson-Importance S&MA Team Deployment; 8) Lesson-Understanding of S&MA In-Line Engineering versus Assurance; 9) Lesson-Importance of Close Coordination between Supportability and Reliability/Maintainability; 10) Lesson-Importance of Engineering Data Systems; 11) Lesson-Importance of Early Development of Supporting Databases; 12) Lesson-Importance of Coordination with Safety Assessment/Review Panels; 13) Lesson-Implementation of Software Reliability; 14) Lesson-Implementation of S&MA Technical Authority/Chief S&MA Officer; 15) Lesson-Importance of S&MA Evaluation of Project Risks; 16) Lesson-Implementation of Critical Items List and Government Mandatory Inspections; 17) Lesson-Implementation of Critical Items List Mandatory Inspections; 18) Lesson-Implementation of Test Article Safety Analysis; and 19) Lesson-Importance of Procurement Quality.

Anderson, Joel M.↗
DOE OSTI2023

SAM Software Quality Assurance Plan Implementation and NQA-1 Assessment

The System Analysis Module (SAM) is an advanced and modern system analysis tool being developed at Argonne National Laboratory under the U.S. DOE Office of Nuclear Energy’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. As a modern-day software, SAM development included efforts to follow best practices in software development. These best practices include version control using git, independent reviews of development activities, and detailed descriptions of developments and bug fixes using the GitLab issue and merge request system. In Fiscal Year 2023 (FY23), the SAM development team set out to formalize a Software Quality Assurance (SQA) program that allowed industry partners to credit the informal steps being taken by the SAM development team to ensure the quality of the software. As part of formalizing an SQA program, a SQA Plan (SQAP) was developed, implemented, and assessed. The SAM SQAP targets compliance with NQA-1-2008/2009 Addenda. The SQAP builds on the MOOSE SQAP and the Argonne SSQAPP while accounting for the needs of the multi-organization SAM development team. Modifications to the SAM repository structure, including a new testing system, updated test cases, and the development of an internal website, facilitate the implementation of the SAM SQAP. The initial assessment of the SAM SQAP indicated that the SAM SQAP was adequately and effectively implemented and additional work was required to improve the compliance of the SAM SQAP with NQA-1 2008/2009 Addenda. Several improvements to the SQAP have already been drafted to address these assessments and future work is planned to further improve the SAM SQAP in support of end-user needs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Evaluating User Errors and Temporal Trends in Marine Fish Communities Using 360-Degree Underwater Photography

The use of environmental DNA (eDNA) sampling has been proposed as a complementary method to monitor fish species in marine environments, offering a non-invasive and potentially more efficient approach to marine species observations. eDNA monitoring could be especially useful in and around sites targeted for marine energy generation as these regions need regular monitoring that would be impractical with traditional techniques. Before we can fully rely upon eDNA, we must first verify its accuracy against other proven methods, such as the use of underwater photography. In this study, I deployed a 360-degree camera in the tidal channel of Sequim Bay once a month during several hours overlapping slack tide. I investigated how having multiple people identify and count fish on underwater images could affect the overall results. Using chi square tests in R, I compared my fish identifications and counts to those made by another intern on the same images recorded in August. I found significant differences in the number of species identified and the total individual counts between the two different datasets. I also tested the statistical differences in both Shannon diversity and Pielou evenness indices between the August, September, and November camera deployments using a Hutcheson t-test. Only one significant difference was found in the Shannon index comparisons, and none were found between the Pielou evenness comparisons. These findings show that if multiple identifiers are used to process underwater images, quality control checks must be made to reduce the potential for error. This also points toward the possibility to leverage more advanced image analysis processes, such as automated image analysis software. The findings from this study also show that the dynamics of marine fish communities can vary over a few months; however, further analysis is needed to determine the extent of the seasonal changes in Sequim Bay.

59 BASIC BIOLOGICAL SCIENCES↗