Results for “confidence measures” — Page 2

DOE OSTI2026

Development and implementation of high-throughput proteomic and metabolomics assays by using advanced chromatographic and mass spectrometric systems (CRADA Final Report)

The mission of this CRADA with Agilent was to couple powerful MS platforms (QQQ, IM-QTOFMS) with Agilent’s novel Ultra-High-Performance Liquid Chromatography (UHPLC) fast metabolomic workflows and perform ABF Machine Learning (ML) to generated datasets. Agilent transferred UHPLC methods to PNNL and LBNL and methods were implemented and demonstrated in both labs, achieving total acquisition times of < 10 min. Metabolites analyzed using Agilent’s shared methods included metabolites from central carbon metabolism, common across hosts, and metabolites unique to engineered strains. Standards were acquired in an UHPLC-Drift Tube Ion Mobility Mass Spectrometer (DTIMS) system for the first time within the context of ABF and methods were optimized based on Agilent’s protocols. Samples from ABF hosts Pseudomonas putida, Aspergillus pseudoterreus, Aspergillus niger and Rhodosporidium toruloides were analyzed using the UHPLC-DTIMS platform for a total of 276 runs. A data analysis workflow compatible with the Experimental Data Depot (EDD) and completely shareable was developed for the acquired UHPLC-DTIMS data. Samples were analyzed using a Data Independent Acquisition Approach (DIA), which for most of the standards provided more transitions therefore increasing detection confidence. Using the data acquired by PNNL, LBNL, and Agilent’s specifications from previous ML projects, SNL applied an ensemble ML strategy to pick the best performing model for automated LC-method selection. Finally, with the contribution of the participant labs and Agilent, SNL developed an Automated Method Selection (AMS) software tool to predict the best liquid chromatography method for analysis of any new molecules of interest. Samples with novel pathways and new metabolite targets of interest are generated at a high pace in the ABF. Overall, the project advanced rapid metabolomics by combining liquid chromatography, ion mobility spectrometry, and data-independent mass spectrometry with machine learning. This multidimensional approach uses retention time, collision cross-section, precursor mass, and fragment-ion information to distinguish chemically similar metabolites that can be difficult to resolve using conventional liquid- or gas-chromatography methods. The resulting workflow also provided automated metabolite-identification error estimates, addressing a recognized need for statistical confidence measures in metabolomics.

Petzold, Christopher [Lawrence Berkeley National L↗

DOE OSTI2023

Six More Ultra-faint Milky Way Companions Discovered in the DECam Local Volume Exploration Survey

We report the discovery of six ultra-faint Milky Way satellites identified through matched-filter searches conducted using Dark Energy Camera (DECam) data processed as part of the second data release of the DECam Local Volume Exploration (DELVE) survey. Leveraging deep Gemini/GMOS-N imaging (for four candidates) as well as follow-up DECam imaging (for two candidates), we characterize the morphologies and stellar populations of these systems. We find that these candidates all share faint absolute magnitudes (M$_{V}$ ≥ −3.2 mag) and old, metal-poor stellar populations (τ > 10 Gyr, [Fe/H] < −1.4 dex). Three of these systems are more extended (r$_{1/2}$ > 15 pc), while the other three are compact (r$_{1/2}$ < 10 pc). From these properties, we infer that the former three systems (Boötes V, Leo Minor I, and Virgo II) are consistent with ultra-faint dwarf galaxy classifications, whereas the latter three (DELVE 3, DELVE 4, and DELVE 5) are likely ultra-faint star clusters. Using data from the Gaia satellite, we confidently measure the proper motion of Boötes V, Leo Minor I, and DELVE 4, and tentatively detect a proper-motion signal from DELVE 3 and DELVE 5; no signal is detected for Virgo II. We use these measurements to explore possible associations between the newly discovered systems and the Sagittarius dwarf spheroidal, the Magellanic Clouds, and the Vast Polar Structure, finding several plausible associations. Our results offer a preview of the numerous ultra-faint stellar systems that will soon be discovered by the Vera C. Rubin Observatory and highlight the challenges of classifying the faintest stellar systems.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS1979

Water vapor differential absorption lidar development and evaluation

A ground-based differential absorption lidar (DIAL) system is described which has been developed for vertical range-resolved measurements of water vapor. The laser transmitter consists of a ruby-pumped dye laser, which is operated on a water vapor absorption line at 724.372 nm. Part of the ruby laser output is transmitted simultaneously with the dye laser output to determine atmospheric scattering and attenuation characteristics. The dye and ruby laser backscattered light is collected by a 0.5-m diam telescope, optically separated in the receiver package, and independently detected using photomultiplier tubes. Measurements of vertical water vapor concentration profiles using the DIAL system at night are discussed, and comparisons are made between the water vapor DIAL measurements and data obtained from locally launched rawinsondes. Agreement between these measurements was found to be within the uncertainty of the rawinsonde data to an altitude of 3 km. Theoretical simulations of this measurement were found to give reasonably accurate predictions of the random error of the DIAL measurements. Confidence in these calculations will permit the design of aircraft and Shuttle DIAL systems and experiments using simulation results as the basis for defining lidar system performance requirements

Browell, E. V.↗

NASA NTRS1981

Energy normalization of TV viewed optical correlation (automated correlation plane analyzer for an optical processor)

An automatic correlation plane processor that can rapidly acquire, identify, and locate the autocorrelation outputs of a bank of multiple optical matched filters is described. The read-only memory (ROM) stored digital silhouette of each image associated with each matched filter allows TV video to be used to collect image energy to provide accurate normalization of autocorrelations. The resulting normalized autocorrelations are independent of the illumination of the matched input. Deviation from unity of a normalized correlation can be used as a confidence measure of correct image identification. Analog preprocessing circuits permit digital conversion and random access memory (RAM) storage of those video signals with the correct amplitude, pulse width, rising slope, and falling slope. TV synchronized addressing of 3 RAMs permits on-line storage of: (1) the maximum unnormalized amplitude, (2) the image x location, and (3) the image y location of the output of each of up to 99 matched filters. A fourth RAM stores all normalized correlations. A normalization approach, normalization for cross correlations, a system's description with block diagrams, and system's applications are discussed.

Grumet, A.↗

NASA NTRS1990

Neural network classification - A Bayesian interpretation

The relationship between minimizing a mean squared error and finding the optimal Bayesian classifier is reviewed. This provides a theoretical interpretation for the process by which neural networks are used in classification. A number of confidence measures are proposed to evaluate the performance of the neural network classifier within a statistical framework.

Wan, Eric A.↗

NASA NTRS1992

Modified fast frequency acquisition via adaptive least squares algorithm

A method and the associated apparatus for estimating the amplitude, frequency, and phase of a signal of interest are presented. The method comprises the following steps: (1) inputting the signal of interest; (2) generating a reference signal with adjustable amplitude, frequency and phase at an output thereof; (3) mixing the signal of interest with the reference signal and a signal 90 deg out of phase with the reference signal to provide a pair of quadrature sample signals comprising respectively a difference between the signal of interest and the reference signal and a difference between the signal of interest and the signal 90 deg out of phase with the reference signal; (4) using the pair of quadrature sample signals to compute estimates of the amplitude, frequency, and phase of an error signal comprising the difference between the signal of interest and the reference signal employing a least squares estimation; (5) adjusting the amplitude, frequency, and phase of the reference signal from the numerically controlled oscillator in a manner which drives the error signal towards zero; and (6) outputting the estimates of the amplitude, frequency, and phase of the error signal in combination with the reference signal to produce a best estimate of the amplitude, frequency, and phase of the signal of interest. The preferred method includes the step of providing the error signal as a real time confidence measure as to the accuracy of the estimates wherein the closer the error signal is to zero, the higher the probability that the estimates are accurate. A matrix in the estimation algorithm provides an estimate of the variance of the estimation error.

Kumar, Rajendra↗

NASA NTRS1993

Interactive and cooperative sensing and control for advanced teleoperation

This paper presents the paradigm of interactive and cooperative sensing and control as a fundamental mechanism of integrating and fusing the strengths of man and machine for advanced teleoperation. The interactive and cooperative sensing and control is considered as an extended and generalized form of traded and shared control. The emphasis of interactive and cooperative sensing and control is given to the distribution of mutually nonexclusive subtasks to man and machine, the interactive invocation of subtasks under the man/machine symbiotic relationship, and the fusion of information and decisionmaking between man and machine according to their confidence measures. The proposed interactive and cooperative sensing and control system is composed of such major functional blocks as the logical sensor system, the sensor-based local autonomy, the virtual environment formation, and the cooperative decision-making between man and machine. The Sensing-Knowledge-Command (SKC) fusion network is proposed as a fundamental architecture for implementing cooperative and interactive sensing and control. Simulation results are shown.

Lee, Sukhan↗

NASA NTRS1992

Assimilation of a knowledge base and physical models to reduce errors in passive-microwave classifications of sea ice

An expert system framework has been developed to classify sea ice types using satellite passive microwave data, an operational classification algorithm, spatial and temporal information, ice types estimated from a dynamic-thermodynamic model, output from a neural network that detects the onset of melt, and knowledge about season and region. The rule base imposes boundary conditions upon the ice classification, modifies parameters in the ice algorithm, determines a `confidence' measure for the classified data, and under certain conditions, replaces the algorithm output with model output. Results demonstrate the potential power of such a system for minimizing overall error in the classification and for providing non-expert data users with a means of assessing the usefulness of the classification results for their applications.

Maslanik, J. A.↗

NASA NTRS1992

Interactive and cooperative sensing and control for advanced teleoperation

This paper presents the paradigm of interactive and cooperative sensing and control as a fundamental mechanism of integrating and fusing the strengths of man and machine for advanced teleoperation. The interactive and cooperative sensing and control is considered as an extended and generalized form of traded and shared control. The emphasis of interactive and cooperative sensing and control is given to the distribution of mutually nonexclusive subtasks to man and machine, the interactive invocation of subtasks under the man/machine symbiotic relationship, and the fusion of information and decision-making between man and machine according to their confidence measures. The proposed interactive and cooperative sensing and control system is composed of such major functional blocks as the logical sensor system, the sensor-based local autonomy, the virtual environment formation, and the cooperative decision-making between man and machine. A case study is performed to demonstrate the feasibility of implementing the fundamental theory and system architecture of interactive and cooperative sensing and control, proposed for the new generation of teleoperation.

Lee, Sukhan↗

NASA NTRS1993

SATCOM simulator speeds MSS deployment and lowers costs

Mobile satellite systems (MSS) are being proposed and licensed at an accelerating rate. How can the design, manufacture, and performance of these systems be optimized at costs that allow a reasonable return on investment? The answer is the use of system simulation techniques beginning early in the system design and continuing through integration, pre- and post-launch monitoring, and in-orbit monitoring. This paper focuses on using commercially available, validated simulation instruments to deliver accurate, repeatable, and cost effective measurements throughout the life of a typical mobile satellite system. A satellite communications test set is discussed that provides complete parametric test capability with a significant improvement in measurement speed for manufacturing, integration, and pre-launch and in-orbit testing. The test set can simulate actual up and down link traffic conditions to evaluate the effects of system impairments, propagation and multipath on bit error rate (BER), channel capacity and transponder and system load balancing. Using a standard set of commercial instruments to deliver accurate, verifiable measurements anywhere in the world speeds deployment, generates measurement confidence, and lowers total system cost.

Carey, Tim↗

NASA NTRS1994

A comparative study of clock rate and drift estimation

Five different methods of drift determination and four different methods of rate determination were compared using months of hourly phase and frequency data from a sample of cesium clocks and active hydrogen masers. Linear least squares on frequency is selected as the optimal method of determining both drift and rate, more on the basis of parameter parsimony and confidence measures than on random and systematic errors.

Breakiron, Lee A.↗

NASA NTRS2003

A Framework for Intelligent Rocket Test Facilities with Smart Sensors

A long-term center goal at the John C. Stennis Space Center (SSC) is the formulation and implementation of a framework for an Intelligent Rocket Test Facility (IRTF), which incorporates distributed smart sensor elements. The IRTF is to provide reliable, high-confident measurements. Specific objectives include: 1. Definition of a framework and architecture that supports implementation of highly autonomous methodologies founded on basic physical principles and embedded knowledge. 2. Modeling of autonomous sensors and processes as self-sufficient, evolutionary elements. 3. Development of appropriate communications protocols to enable the complex interactions that must take place to allow timely and high-quality flow of of information among all the autonomous elements of the system. 4. Development of lab-scale prototypes of key system elements. Though our application is next-generation rocket test facilities, applications for the approach are much wider and include monitoring of shuttle launch operations, air and spacecraft operations and health monitoring, and other large-scale industrial system operations such as found in processing and manufacturing plans. Elements of prototype IRTF have been implemented in preparation for advanced development and validation using rocket test stand facilities as SSC. This work has identified issues that are important to further development of complex network and should be of interest to other working with sensor networks.

Figueroa, Fernando↗

NASA NTRS2005

Kalman Filter Constraint Tuning for Turbofan Engine Health Estimation

Kalman filters are often used to estimate the state variables of a dynamic system. However, in the application of Kalman filters some known signal information is often either ignored or dealt with heuristically. For instance, state variable constraints are often neglected because they do not fit easily into the structure of the Kalman filter. Recently published work has shown a new method for incorporating state variable inequality constraints in the Kalman filter, which has been shown to generally improve the filter s estimation accuracy. However, the incorporation of inequality constraints poses some risk to the estimation accuracy as the Kalman filter is theoretically optimal. This paper proposes a way to tune the filter constraints so that the state estimates follow the unconstrained (theoretically optimal) filter when the confidence in the unconstrained filter is high. When confidence in the unconstrained filter is not so high, then we use our heuristic knowledge to constrain the state estimates. The confidence measure is based on the agreement of measurement residuals with their theoretical values. The algorithm is demonstrated on a linearized simulation of a turbofan engine to estimate engine health.

Simon, Dan↗

NASA NTRS2012

Health-Enabled Smart Sensor Fusion Technology

A process was designed to fuse data from multiple sensors in order to make a more accurate estimation of the environment and overall health in an intelligent rocket test facility (IRTF), to provide reliable, high-confidence measurements for a variety of propulsion test articles. The object of the technology is to provide sensor fusion based on a distributed architecture. Specifically, the fusion technology is intended to succeed in providing health condition monitoring capability at the intelligent transceiver, such as RF signal strength, battery reading, computing resource monitoring, and sensor data reading. The technology also provides analytic and diagnostic intelligence at the intelligent transceiver, enhancing the IEEE 1451.x-based standard for sensor data management and distributions, as well as providing appropriate communications protocols to enable complex interactions to support timely and high-quality flow of information among the system elements.

Wang, Ray↗

NASA NTRS2020

Ground and In-Flight Calibration of the OSIRIS-REx Camera Suite

The OSIRIS-REx Camera Suite (OCAMS) onboard the OSIRIS-REx spacecraft is used to study the shape and surface of the mission’s target, asteroid (101955) Bennu,in support of the selection of a sampling site. We present calibration methods and results for the three OCAMS cameras—MapCam, PolyCam, and SamCam—using data from pre-flight and in-flight calibration campaigns. Pre-flight calibrations established a baseline fora variety of camera properties, including bias and dark behavior, flat fields, stray light, and radiometric calibration. In-flight activities updated these calibrations where possible, allowing us to confidently measure Bennu’s surface. Accurate calibration is critical not only for establishing a global understanding of Bennu, but also for enabling analyses of potential sampling locations and for providing scientific context for the returned sample.

E Mazarico↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek A Whitman↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek Whitman↗

DOE OSTI2021

Super Resolution Digital Image Correlation (SR-DIC): an Alternative to Image Stitching at High Magnifications

Abstract Background High-resolution Digital Image Correlation (DIC) measurements have previously been produced by stitching of neighboring images, which often requires short working distances. Separately, the image processing community has developed super resolution (SR) imaging techniques, which improve resolution by combining multiple overlapping images. Objective This work investigates the novel pairing of super resolution with digital image correlation, as an alternative method to produce high-resolution full-field strain measurements. Methods First, an image reconstruction test is performed, comparing the ability of three previously published SR algorithms to replicate a high-resolution image. Second, an applied translation is compared against DIC measurement using both low- and super-resolution images. Third, a ring sample is mechanically deformed and DIC strain measurements from low- and super-resolution images are compared. Results SR measurements show improvements compared to low-resolution images, although they do not perfectly replicate the high-resolution image. SR-DIC demonstrates reduced error and improved confidence in measuring rigid body translation when compared to low resolution alternatives, and it also shows improvement in spatial resolution for strain measurements of ring deformation. Conclusions Super resolution imaging can be effectively paired with Digital Image Correlation, offering improved spatial resolution, reduced error, and increased measurement confidence.

Hansen, R. S.↗