Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “error impact analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

Comparison of Test and Finite Element Analysis for Two Full-Scale Helicopter Crash Tests

Finite element analyses have been performed for two full-scale crash tests of an MD-500 helicopter. The first crash test was conducted to evaluate the performance of a composite deployable energy absorber under combined flight loads. In the second crash test, the energy absorber was removed to establish the baseline loads. The use of an energy absorbing device reduced the impact acceleration levels by a factor of three. Accelerations and kinematic data collected from the crash tests were compared to analytical results. Details of the full-scale crash tests and development of the system-integrated finite element model are briefly described along with direct comparisons of acceleration magnitudes and durations for the first full-scale crash test. Because load levels were significantly different between tests, models developed for the purposes of predicting the overall system response with external energy absorbers were not adequate under more severe conditions seen in the second crash test. Relative error comparisons were inadequate to guide model calibration. A newly developed model calibration approach that includes uncertainty estimation, parameter sensitivity, impact shape orthogonality, and numerical optimization was used for the second full-scale crash test. The calibrated parameter set reduced 2-norm prediction error by 51% but did not improve impact shape orthogonality.

Annett, Martin S.↗

Lensing without borders – I. A blind comparison of the amplitude of galaxy–galaxy lensing between independent imaging surveys

ABSTRACT Lensing without borders is a cross-survey collaboration created to assess the consistency of galaxy–galaxy lensing signals (ΔΣ) across different data sets and to carry out end-to-end tests of systematic errors. We perform a blind comparison of the amplitude of ΔΣ using lens samples from BOSS and six independent lensing surveys. We find good agreement between empirically estimated and reported systematic errors which agree to better than 2.3σ in four lens bins and three radial ranges. For lenses with zL > 0.43 and considering statistical errors, we detect a 3–4σ correlation between lensing amplitude and survey depth. This correlation could arise from the increasing impact at higher redshift of unrecognized galaxy blends on shear calibration and imperfections in photometric redshift calibration. At zL > 0.54, amplitudes may additionally correlate with foreground stellar density. The amplitude of these trends is within survey-defined systematic error budgets that are designed to include known shear and redshift calibration uncertainty. Using a fully empirical and conservative method, we do not find evidence for large unknown systematics. Systematic errors greater than 15 per cent (25 per cent) ruled out in three lens bins at 68 per cent (95 per cent) confidence at z < 0.54. Differences with respect to predictions based on clustering are observed to be at the 20–30 per cent level. Our results therefore suggest that lensing systematics alone are unlikely to fully explain the ‘lensing is low’ effect at z < 0.54. This analysis demonstrates the power of cross-survey comparisons and provides a promising path for identifying and reducing systematics in future lensing analyses.

79 ASTRONOMY AND ASTROPHYSICS↗

The Uniform Methods Project: Smart Thermostat Evaluation Protocol

A smart thermostat is an internet-connected device that controls home heating, ventilation, and air-conditioning (HVAC) equipment and can automatically adjust temperature set points to optimize performance and achieve energy savings. Smart thermostat features often include two way communication, occupancy detection (such as geofencing and occupancy sensors), schedule learning, and seasonal optimization algorithms. Smart thermostats can control most conventional HVAC systems, including central air conditioners, heat pumps, and forced air furnaces. Several types of residential utility programs offer smart thermostats as replacements measures. Working with smart thermostat vendors, utilities can offer separate optimization programs to produce energy savings beyond those achieved by installing a smart thermostat. From an evaluation perspective, smart thermostat programs have several noteworthy features. First, the energy savings from a smart thermostat may change over the life of the device. As a smart thermostat is connected to the internet, original equipment manufacturers can update the thermostat software to improve the thermostat's energy efficiency. Likewise, users can adjust the thermostat settings and schedules over time in response to changes in weather, thermal comfort, energy prices, or preferences for energy efficiency. Additionally, many thermostat manufacturers offer seasonal optimization programs that recommend changes or make minor, automated adjustments to the thermostat settings to improve energy efficiency. These opt-in programs are now standard offerings for many smart thermostat manufacturers and provided at no additional cost to users. The potential for software updates and continuous optimization and the evolving nature of user interactions mean future energy savings may differ from first-year savings and the energy savings of smart thermostats may need to be evaluated more than once. Second, smart thermostats often have small unit energy savings relative to a home's total energy consumption, especially in comparison to whole- home retrofit programs. This can make it difficult to detect the smart thermostat savings in billing or advanced metering infrastructure (AMI) meter consumption data. For example, as cooling loads in many regions average about 20% of annual electricity consumption, smart thermostat savings of 10% of cooling energy use would equate to a 2% reduction in home electricity consumption. Evaluators should use regression analysis of whole-home billing consumption or advanced metering infrastructure (AMI) meter consumption data to evaluate smart thermostat savings because, as explained at greater length below , these data are usually available to evaluators and regression can control for the impacts of weather and other potentially confounding factors on a home's energy consumption. Finally, as with other energy efficiency programs, participation in smart thermostat programs is self-selective. As discussed at greater length below , smart thermostat participants tend to be, among other things, younger, higher-income, and more likely to adopt electric vehicles (EVs) and internet connected devices than nonparticipants. These differences are often unobservable to the evaluator and correlated with a home's energy consumption, creating the potential for bias in estimating savings. Due to the small unit savings of thermostats, errors and biases from self-selection that may not be very consequential when evaluating a whole- home retrofits (e.g., ±2% of home electricity consumption) can have a major impact when evaluating the savings and cost-effectiveness of smart thermostat programs. A percentage point change in the estimated savings could affect the cost-effectiveness of a program. This means it is important for evaluators to assess and to minimize the potential for error from selection bias in estimating smart thermostat program savings. The Uniform Methods Project provides model protocols for determining energy savings and demand reductions that result from specific energy efficiency measures implemented through state and utility programs. In most cases, the measure protocols are based on a particular option identified by the International Performance Verification and Measurement Protocol ; however, this work provides a more detailed approach to implementing that option. Each chapter is written by technical experts in collaboration with their peers, reviewed by industry experts, and subject to public review and comment. The UMP protocols can be used by utilities, program administrators, public utility commissions, evaluators, and other stakeholders for both program planning and evaluation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The Use of Red Green Blue Air Mass Imagery to Investigate the Role of Stratospheric Air in a Non-convective Wind Event

Non-convective wind events commonly occur with passing extratropical cyclones and have significant societal and economic impacts. Since non-convective winds often occur in the absence of specific phenomena such as a thunderstorm, tornado, or hurricane, the public are less likely to heed high wind warnings and continue daily activities. Thus non-convective wind events result in as many fatalities as straight line thunderstorm winds. One physical explanation for non-convective winds includes tropopause folds. Improved model representation of stratospheric air and associated non-convective wind events could improve non-convective wind forecasts and associated warnings. In recent years, satellite data assimilation has improved skill in forecasting extratropical cyclones; however errors still remain in forecasting the position and strength of extratropical cyclones as well as the tropopause folding process. The goal of this study is to determine the impact of assimilating satellite temperature and moisture retrieved profiles from hyperspectral infrared (IR) sounders (i.e. Atmospheric Infrared Sounder (AIRS), Cross-track Infrared and Microwave Sounding Suite (CrIMSS), and Infrared Atmospheric Sounding Interferometer (IASI)) on the model representation of the tropopause fold and an associated high wind event that impacted the Northeast United States on 09 February 2013. Model simulations using the Advanced Research Weather Research and Forecasting Model (ARW) were conducted on a 12-km grid with cycled data assimilation mimicking the operational North American Model (NAM). The results from the satellite assimilation run are compared to a control experiment (without hyperspectral IR retrievals), Modern Era-Retrospective Analysis for Research and Applications (MERRA) reanalysis, and Rapid Refresh analyses.

Berndt, E. B.↗

The Impact of the Assimilation of Hyperspectral Infrared Retrieved Profiles on Advanced Weather and Research Model Simulations of a Non-Convective Wind Event

Non-convective wind events commonly occur with passing extratropical cyclones and have significant societal and economic impacts. Since non-convective winds often occur in the absence of specific phenomena such as a thunderstorm, tornado, or hurricane, the public are less likely to heed high wind warnings and continue daily activities. Thus non-convective wind events result in as many fatalities as straight line thunderstorm winds. One physical explanation for non-convective winds includes tropopause folds. Improved model representation of stratospheric air and associated non-convective wind events could improve non-convective wind forecasts and associated warnings. In recent years, satellite data assimilation has improved skill in forecasting extratropical cyclones; however errors still remain in forecasting the position and strength of extratropical cyclones as well as the tropopause folding process. The goal of this study is to determine the impact of assimilating satellite temperature and moisture retrieved profiles from hyperspectral infrared (IR) sounders (i.e. Atmospheric Infrared Sounder (AIRS), Cross-track Infrared and Microwave Sounding Suite (CrIMSS), and Infrared Atmospheric Sounding Interferometer (IASI)) on the model representation of the tropopause fold and an associated high wind event that impacted the Northeast United States on 09 February 2013. Model simulations using the Advanced Research Weather Research and Forecasting Model (ARW) were conducted on a 12-km grid with cycled data assimilation mimicking the operational North American Model (NAM). The results from the satellite assimilation run are compared to a control experiment (without hyperspectral IR retrievals), Modern Era-Retrospective Analysis for Research and Applications (MERRA) reanalysis, and Rapid Refresh analyses.

Berndt, Emily↗

Impact of the Assimilation of Hyperspectral Infrared Retrieved Profiles on Advanced Weather and Research Model Simulations of a Non-Convective Wind Event

Non-convective wind events commonly occur with passing extratropical cyclones and have significant societal and economic impacts. Since non-convective winds often occur in the absence of specific phenomena such as a thunderstorm, tornado, or hurricane, the public are less likely to heed high wind warnings and continue daily activities. Thus non-convective wind events result in as many fatalities as straight line thunderstorm winds. One physical explanation for non-convective winds includes tropopause folds. Improved model representation of stratospheric air and associated non-convective wind events could improve non-convective wind forecasts and associated warnings. In recent years, satellite data assimilation has improved skill in forecasting extratropical cyclones; however errors still remain in forecasting the position and strength of extratropical cyclones as well as the tropopause folding process. The goal of this study is to determine the impact of assimilating satellite temperature and moisture retrieved profiles from hyperspectral infrared (IR) sounders (i.e. Atmospheric Infrared Sounder (AIRS), Cross-track Infrared and Microwave Sounding Suite (CrIMSS), and Infrared Atmospheric Sounding Interferometer (IASI)) on the model representation of the tropopause fold and an associated high wind event that impacted the Northeast United States on 09 February 2013. Model simulations using the Advanced Research Weather Research and Forecasting Model (ARW) were conducted on a 12-km grid with cycled data assimilation mimicking the operational North American Model (NAM). The results from the satellite assimilation run are compared to a control experiment (without hyperspectral IR retrievals), Modern Era-Retrospective Analysis for Research and Applications (MERRA) reanalysis, and Rapid Refresh analyses.

Berndt, E. B.↗

Vicarious Calibration of the Long Near Infrared Band: Cross-Sensor Differences in Sensitivity

Numerous assumptions and approximations are employed when translating satellite-derived radiance to surface remote sensing reflectance (RRS) for ocean color applications. Among these is the vicarious calibration coefficient (g) of the “long” near infrared band (NIRL) used for atmospheric correction. For this band, the prelaunch calibration has always been deemed sufficient [thus g(NIRL) = 1.00] as long as other bands are vicariously calibrated. Recent research, however, suggests that Moderate Resolution Imaging Spectroradiometer (MODIS)/Aqua RRS time series is quite sensitive to g(NIRL) (and associated vicarious gains in other bands). In this work, we assessed the sensitivity of Visible Infrared Imaging Radiometer Suite onboard the Suomi National Polar-orbiting Partnership satellite (VIIRS/SNPP) RRS to NIRL calibration and compared our results to previous MODIS/Aqua and Sea-viewing Wide Field-of-View Sensor onboard OrbView2 (SeaWiFS)/OrbView2 analysis. In doing so, we note that g(NIRL) sensitivities of mission-averaged RRS time series are lower for VIIRS and Sea-WiFS, relative to MODIS. At the scale of monthly climatologies (MCs), however, all sensors show prominent g(NIRL) sensitivity with that of SeaWiFS being the most substantial. These findings informed simulation analyses, whereby we identified signal-tonoise ratio (SNR) and radiant path geometry, as well as their interaction, as having notable impacts on g(NIRL) sensitivity. As such, g(NIRL) sensitivity is a necessary consideration for reflectance uncertainty budgets, especially for sensors with higher NIR SNR or particular prevailing radiant path geometries. Given the geometry components embedded within g(NIRL) sensitivity, such studies should be coupled with cross-sensor intercalibrations [e.g., using simultaneous same view (SSV) measurements] toward minimizing NIRL errors between satellite instruments, but such efforts will not completely remediate remaining cross-sensor biases in RRS. Index Terms—Moderate Resolution Imaging Spectroradiometer (MODIS)/Aqua, ocean color, ocean gyres, Sea-viewing Wide Field-of-View Sensor onboard OrbView2 (SeaWiFS), system vicarious calibration (SVC), Visible Infrared Imaging Radiometer Suite onboard the Suomi National Polar-orbiting Partnership satellite (VIIRS/SNPP).

MODIS↗

Modeling impurity concentrations in liquid argon detectors

Impurities in noble liquid detectors used for neutrino and dark matter experiments can significantly impact the quality of data. We present an experimentally verified model for describing the dynamics of impurity distributions in liquid argon (LAr) detectors. The model considers sources, sinks, and transport of impurities within and between the gas and liquid argon phases. Measurements of oxygen concentrations in a 20-L LAr multi-purpose test stand are compared to calculations made with this model to show that an accurate description of the concentrations under various operational conditions can be obtained. A result of this analysis is a determination of Henry’s coefficient for oxygen in LAr. Additionally, these calculations also show that some processes have small effects on the impurity dynamics and excluding them yields a solution as a sum of two exponential terms. This solution provides a simple way to extract Henry’s coefficient with negligible approximation error. It is applied to the data and the Henry’s coefficient for oxygen in LAr is obtained as 0.84 +0.09 -0.05 , consistent with literature results. Finally, based on the analysis of the data with the model, we further suggest that, for a large liquid argon detector, barriers to flow (“baffles”) installed in the gas phase to restrict flow can help reduce the ultimate impurity concentration in the LAr.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Neural Network Prediction of Failure of Damaged Composite Pressure Vessels from Strain Field Data Acquired by a Computer Vision Method

This effort used a new and novel method of acquiring strains called Sub-pixel Digital Video Image Correlation (SDVIC) on impact damaged Kevlar/epoxy filament wound pressure vessels during a proof test. To predict the burst pressure, the hoop strain field distribution around the impact location from three vessels was used to train a neural network. The network was then tested on additional pressure vessels. Several variations on the network were tried. The best results were obtained using a single hidden layer. SDVIC is a fill-field non-contact computer vision technique which provides in-plane deformation and strain data over a load differential. This method was used to determine hoop and axial displacements, hoop and axial linear strains, the in-plane shear strains and rotations in the regions surrounding impact sites in filament wound pressure vessels (FWPV) during proof loading by internal pressurization. The relationship between these deformation measurement values and the remaining life of the pressure vessels, however, requires a complex theoretical model or numerical simulation. Both of these techniques are time consuming and complicated. Previous results using neural network methods had been successful in predicting the burst pressure for graphite/epoxy pressure vessels based upon acoustic emission (AE) measurements in similar tests. The neural network associates the character of the AE amplitude distribution, which depends upon the extent of impact damage, with the burst pressure. Similarly, higher amounts of impact damage are theorized to cause a higher amount of strain concentration in the damage effected zone at a given pressure and result in lower burst pressures. This relationship suggests that a neural network might be able to find an empirical relationship between the SDVIC strain field data and the burst pressure, analogous to the AE method, with greater speed and simplicity than theoretical or finite element modeling. The process of testing SDVIC neural network analysis and some encouraging preliminary results are presented in this paper. Details are given concerning the processing of SDVIC output data such that it may be used as back propagation neural network (BPNN) input data. The software written to perform this processing and the BPNN algorithm are also discussed. It will be shown that, with limited training, test results indicate an average error in burst pressure prediction of approximately six percent,

Russell, Samuel S.↗

Towards Precision Photometric Type ia Supernova Cosmology With Machine Learning

The revolutionary discovery of dark energy and accelerating cosmic expansion was made with just 42 type Ia supernovae (SNe Ia) in 1999. Since then, large synoptic surveys, e.g., Dark Energy Survey (DES), have observed thousands more SNe Ia and the upcoming Rubin Legacy Survey of Space and Time (LSST) and Roman Space Telescope promise to deliver millions in the next decade. This unprecedented data volume can produce the required precision to unambiguously test concordance cosmology which could represent a monumental shift in our understanding of dark energy and its role in cosmic history. However, extracting a pure SN Ia sample with accurate redshifts for such a large dataset will be a challenge. Specifically, spectroscopic classification will not be possible for the vast majority of discovered objects, and only ~25% will have spectroscopic redshifts. This thesis presents a series of observational and methodological studies designed to address the questions associated with this new era of photometric SN Ia cosmology. First, we present a machine learning method for photometric classification of SNe, Supernova Classification with a COnvolutional Neural Network (SCONE). Photometric classification enables SNe with no spectroscopic information to be confidently categorized, and is a critical component of current and future analysis pipelines. SCONE achieves >99% accuracy distinguishing simulated SNe Ia from non-Ia SNe, and has been integrated into DES, LSST, and Roman analysis pipelines. We also demonstrate the efficacy of SCONE on early-time photometric classification, which will be vital for optimal allocation of spectroscopic resources. We show that SCONE can distinguish between 6 SN types with 75% accuracy on the night of initial discovery, comparable to results in the literature for full-phase SNe. Next, we study current methods for estimating SN Ia redshifts and propose a machine learning alternative that uses SN photometry alone to extract redshift information. Most SNe Ia inherit redshift information from their host galaxy, but the process of matching SNe to the correct host galaxy can be challenging. We systematically analyze the impact of incorrect redshifts from host galaxy mismatch on 5 years of DES SN data, and conclude that improved host matching or redshift estimation methods can reduce our systematic errors by ~10%. In response to this finding, we present a SN photometry-only method for estimating redshifts independent of host galaxy information, Photo-zSNthesis. We show that Photo-zSNthesis redshift estimates are accurate to within 2% across the full redshift range of LSST, a first in the literature. Finally, we focus on the robustness of machine learning (ML) algorithms for real-world and scientific applications. ML models generalize poorly beyond their training set and often experience severe performance degradation when deployed on new data. We demonstrate a general method for improving robustness that achieves new state-of-the-art results on astronomical object classification, wildlife identification, and tumor detection.

79 ASTRONOMY AND ASTROPHYSICS↗

The OpenSE Cookbook: A Practical, Recipe Based Collection of Patterns, Procedures, and Best Practices for Executable Systems Engineering for the Thirty Meter Telescope

The OpenSE Cookbook is an open-sourced collection of patterns, procedures, and best practices targeted for systems engineers who seek guidance on applying model-based and executable systems engineering (MBSE) using SysML. Its content has emerged from the system level modeling effort on the European Framework Program 6 (FP6) and the Thirty Meter Telescope (TMT). The TMT MBSE approach applied the Executable Systems Engineering Method (ESEM) and the open-source Engineering Environment (OpenMBEE) to specify, analyze, and verify requirements of TMT’s Alignment and Phasing System (APS) and the Narrow Field Infrared Adaptive Optics System (NFIRAOS). In these applications, implicit dependencies are made explicit in a formal model through the use of ESEM, OpenMBEE, and SysML modeling constructs. The value proposition for applying this MBSE approach was to establish precise requirements and fine-grained traceability to system designs, and to verify key requirements beginning early in development. The integration of ESEM and the OpenMBEE tooling infrastructure (providing linked-data and web-operability) is a significant added value for the MBSE approach. The APS is responsible for the overall pre-adaptive optics wavefront quality, using starlight to measure wavefront errors and align the TMT optics. In the formally integrated and executable SysML model, simulations are performed to analyze the impact of changed requirements and verify specified constraints for various operational scenarios. The APS team used several modeling patterns to capture information such as the requirements, the operational scenarios, involved subsystems and their interaction points, the estimated or required time durations, and the mass and power consumption. Adaptive optics systems are designed to sense real-time atmospheric turbulence and correct the telescope’s optical beam to remove its effect. The system model for the adaptive optics operational modes was developed to capture sequence behaviors and operational scenarios to run Monte-Carlo simulations for verifying acquisition time, observing efficiency, and operational behavior requirements. The model is particularly useful for investigating the effect of parallelization, identifying interface issues, and re-ordering sequence acquisition tasks. A former version of the Cookbook (which is now updated to MBSE challenges, goals, and lessons learned) included modeling guidelines and conventions for all system aspects, hierarchy levels, and views, which were developed during for the Active Phasing Experiment (APE), an opto-mechatronical system technology demonstrator for the Extremely Large Telescope (ELT). The Cookbook utilizes the above mentioned system models as real-world case-studies to demonstrate and document the applications of the recipes, providing also instructional examples and addressing the available tooling support. The Cookbook is accompanied by a number of SysML models and aodel libraries which facilitate model authoring and maintenance. The Cookbook covers the different aspects of Systems Engineering such as management of Requirements, Design (behavior and structure), Interfaces, Interdisciplinary Integration, Analysis, Trade Studies, and Technical Resources. This paper presents the background, motivation, architecture, and highlights some key content of the Cookbook. For example, interface management, error budget management, requirements verification, Monte Carlo driven analysis, and timing analysis of operational scenarios. The paper discusses how the capabilities of OpenMBEE contributed significantly to the adoption of executable systems engineering.

Brower, Eric↗

Variational Assimilation of Global Microwave Rainfall Retrievals: Physical and Dynamical Impact on GEOS Analyses and Forecasts

Global microwave rainfall retrievals from a 5-satellite constellation, including TMI from TRMM, SSWI from DMSP F13, F14 and F15, and AMSR-E from EOS-AQUA, are assimilated into the NASA Goddard Earth Observing System (GEOS) Data Assimilation System (DAS) using a 1-D variational continuous assimilation (VCA) algorithm. The physical and dynamical impact of rainfall assimilation on GEOS analyses and forecasts is examined at various temporal and spatial scales. This study demonstrates that the 1-D VCA algorithm, which was originally developed and evaluated for rainfall assimilations over tropical oceans, can effectively assimilate satellite microwave rainfall retrievals and improve GEOS analyses over both the Tropics and the extratropics where the atmospheric processes are dominated by different large-scale dynamics and moist physics, and also over the land, where rainfall estimates from passive microwave radiometers are believed to be less accurate. Results show that rainfall assimilation renders the GEOS analysis physically and dynamically more consistent with the observed precipitation at the monthly-mean and 6-hour time scales. Over regions where the model precipitation tends to misbehave in distinctly different rainy regimes, the 1-D VCA algorithm, by compensating for errors in the model s moist time-tendency in a 6-h analysis window, is able to bring the rainfall analysis closer to the observed. The radiation and cloud fields also tend to be in better agreement with independent satellite observations in the rainfall-assimilation m especially over regions where rainfall analyses indicate large improvements. Assimilation experiments with and without rainfall data for a midlatitude frontal system clearly indicates that the GEOS analysis is improved through changes in the thermodynamic and dynamic fields that respond to the rainfall assimilation. The synoptic structures of temperature, moisture, winds, divergence, and vertical motion, as well as vorticity are more realistically captured across the front. Short-term forecasts using initial conditions assimilated with rainfall data also show slight improvements. 1

Lin, Xin↗

Interface Consistency: Phase I Results & Phase II Status

Future exploration missions will rely on designing and developing vehicles and complex systems from within NASA and through multiple external commercial partners to meet mission goals. Despite existing consistency-related agency requirements, NASA’s approach to commercial spaceflight development encourages providers’ flexibility and innovation. This strategy is resulting in significant design diversity across Artemis vehicles. Design best practices and guidelines champion interface consistency to promote mental model development and knowledge transfer. However, research investigating the benefits of consistency is mixed, and little is known about its role in complex systems. Determining the level of risk that system diversity presents is difficult, as there is no established method for quantifying the degree of consistency within and across interfaces, nor is there information about the differential impacts of different types of inconsistency. Phase I of this project (Characterization and Measurement) served as a starting point to better understand the construct of consistency, its application, and the range of studies and methods for measuring it. The project team created a taxonomy of consistency to apply to interfaces as a framework to guide the development of tools to assess intersystem consistency. Checklist and cognitive walkthrough methods were developed for use by human factors (HF) and human-computer interaction (HCI) experts. The Intersystem Consistency Scale (ICS) was developed for interface evaluations with crew. A pilot study evaluated the methods’ ability to distinguish differences between Artemis-like prototype pairs exhibiting either high or low design consistency. In addition, click errors and time on task were collected within the ICS (crew-like) group. Results from our exploratory analysis and lessons learned from the pilot study will be discussed. The project team will also present the status of Phase II (Risk Assessment, Standards and Guidelines). This includes incorporating feedback to redesign the assessment tools, and inputs from displays and training Subject Matter Experts to update tasks and prototype designs. The team will present the risk assessment study design to identify the types and levels of inconsistency that pose the greatest risk to performance. Plans to apply these results toward agency standards and guideline recommendations will also be discussed.

Human-Computer Interaction↗

The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey quasar sample: testing observational systematics on the Baryon Acoustic Oscillation measurement

ABSTRACT Baryon Acoustic Oscillations (BAO) are considered to be a very robust standard ruler against various systematics. This premise has been tested against observational systematics, but not to the level required for the next generation of galaxy surveys such as the Dark Energy Spectroscopic Instrument (DESI) and Euclid. In this paper, we investigate the effect of observational systematics on the BAO measurement of the final sample of quasars from the extended Baryon Oscillation Spectroscopic Survey Data Release 16 in order to prepare and hone a similar analysis for upcoming surveys. We employ catalogues with various treatments of imaging systematic effects using linear and neural network-based non-linear approaches and consider how the BAO measurement changes. We also test how the variations to the BAO fitting model respond to the observational systematics. As expected, we confirm that the BAO measurements obtained from the DR16 quasar sample are robust against imaging systematics well within the statistical error, while reporting slightly modified constraints that shift the line-of-sight BAO signal by less than 1.1 per cent. We use realistic simulations with similar redshift and angular distributions as the DR16 sample to conduct statistical tests for validating the pipeline, quantifying the significance of differences, and estimating the expected bias on the BAO scale in future high-precision data sets. Although we find a marginal impact for the eBOSS QSO data, the work presented here is of vital importance for constraining the nature of dark energy with the BAO feature in the new era of big data cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗

Size distribution and scattering phase function of aerosol particles retrieved from sky brightness measurements

Ground-based measurements of the solar transmission and sky radiance in a horizontal plane through the Sun are taken in several geographical regions and aerosol types: dust in a desert transition zone in Israel, sulfate particles in Eastern and Western Europe, tropical aerosol in Brazil, and mixed continental/maritime aerosol in California. Stratospheric aerosol was introduced after the eruption of Mount Pinatubo in June 1991. Therefore measurements taken before the eruption are used to analyze the properties of tropospheric aerosol; measurements from 1992 are also used to detect the particle size and concentration of stratospheric aerosol. The measurements are used to retrieve the size distribution and the scattering phase function at large scattering angles of the undisturbed aerosol particles. The retrieved properties represent an average on the entire atmospheric column. A comparison between the retrieved phase function for a scattering angle of 120 deg, with phase function predicted from the retrieved size distribution, is used to test the assumption of particle homogeneity and sphericity in radiative transfer models (Mie theory). The effect was found to be small (20% +/- 15%). For the stratospheric aerosol (sulfates), as expected, the phase function was very well predicted using the Mie theory. A model with a power law distribution, based on the spectral dependence of the optical thickness, alpha, cannot estimate accurately the phase function (up to 50% error for lambda = 0.87 microns). Before the Pinatubo eruption the ratio between the volumes of sulfate and coarse particles was very well correlated with alpha. The Pinatubo stratospheric aerosol destroyed this correlation. The aerosol optical properties are compared with analysis of the size, shape, and composition of the individual particles by electron microscopy of in situ samples. The measured volume size distribution before the injection of stratospheric aerosol consistently show two modes, sulfate particles with r(sub m) less than 0.2 microns and coarse paritcles with r(sub m) greater than 0.7 microns. The 'window' in the tropospheric aerosol in this radius range was used to observe a stable stratospheric aerosol in 1992, with r(sub m) approximately 0.5 microns. A combination of such optical thickness and sky measurements can be used to assess the direct forcing and the climatic impact of aerosol. Systematic inversion for the key aerosol types (sulfates, smoke, dust, and maritime aerosol) of the size distribution and phase function can give the relationship between the aerosol physical and optical properties that can be used to compute the radiative forcing. This forcing can be validated in dedicated field experiments.

Kaufman, Y. J.↗

Air Traffic Management-eXploration Testbed for Urban Air Mobility Research and Development

The presentation will describe the architecture, current capabilities and some future enhancements of the testbed that is being developed at the National Aeronautics and Space Administration (NASA) to enable benefit, impact, safety and cost assessments for accelerating the deployment of air traffic management concept and technologies in the national airspace system. The testbed will support analysis of operational feasibility of urban air mobility operations, a part of NASA's Air Traffic Management eXploration project, and provide the data needed by regulatory agencies charged with public safety. Introduction of concepts and technologies, especially new concepts and technologies, is difficult and often takes decades because of the inability to assess the operational impact of the interaction between the proposed concept and technology and operationally deployed systems in terms of system-wide safety, traffic flow efficiency, roles and workload of controllers and traffic managers, and impact on airlines and other operators. To overcome these limitations, the testbed is developing infrastructure to enable mathematical modeling, human-in-the-loop evaluations and testing with operational systems in a simulated environment. In addition to the difficulty of establishing communications between geographically distributed systems, downloading/installing software, and management of startup, error-handling and shutdown, a major impediment for conducting simulations and human-in-the-loop testing with operational systems is the tedious manual scenario generation process. Several of these difficulties have been addressed in the current state of the testbed. The testbed can be described in terms of the following elements (1) web-based frontend and backend, (2) Testbed Builder, (3) Data Distribution Service, (4) Component Library, (5) Simulation Management, and (6) Scenario Generation. The web-based frontend and backend enable the user to interact with the testbed for tasks such as composing a simulation, running a simulation and retrieving output data. The Testbed Builder application launched from the web frontend is a graphical user interface for the user to drag-and-drop and connect predefined blocks for composing a simulation/scenario generation task. The Builder writes a set of instructions for Simulation Management based on the links between the blocks and the block properties such as the component (executable) associated with a particular block. Management of the distributed simulation is accomplished by Execution and Component Managers. Execution Manager interprets the instructions provided by the Builder to instruct the Component Managers to download components from the Component Library to specified computers and to start them up. Once started, the components communicate with each other by publishing messages and subscribing to messages that are delivered by the Data Distribution Service. The Scenario Generation capability can be used for creating traffic scenarios for Multi-Aircraft Control System, which has been used extensively at NASA for human-in-the-loop-based concept evaluations. The presentation will provide a testbed enabled example scenario of Multi-Aircraft Control System based simulation in which the urban air mobility pilot using the conflict detection and resolution system would interact with the air traffic controllers for resolving conflicts with other aircraft during terminal area operations.

Testbed↗

Air Traffic Management-eXploration Testbed for Urban Air Mobility Research and Development

The presentation will describe the architecture, current capabilities and some future enhancements of the testbed that is being developed at the National Aeronautics and Space Administration (NASA) to enable benefit, impact, safety and cost assessments for accelerating the deployment of air traffic management concept and technologies in the national airspace system. The testbed will support analysis of operational feasibility of urban air mobility operations, a part of NASA's Air Traffic Management eXploration project, and provide the data needed by regulatory agencies charged with public safety. Introduction of concepts and technologies, especially new concepts and technologies, is difficult and often takes decades because of the inability to assess the operational impact of the interaction between the proposed concept and technology and operationally deployed systems in terms of system-wide safety, traffic flow efficiency, roles and workload of controllers and traffic managers, and impact on airlines and other operators. To overcome these limitations, the testbed is developing infrastructure to enable mathematical modeling, human-in-the-loop evaluations and testing with operational systems in a simulated environment. In addition to the difficulty of establishing communications between geographically distributed systems, downloading/installing software, and management of startup, error-handling and shutdown, a major impediment for conducting simulations and human-in-the-loop testing with operational systems is the tedious manual scenario generation process. Several of these difficulties have been addressed in the current state of the testbed. The testbed can be described in terms of the following elements- (1) web-based frontend and backend, (2) Testbed Builder, (3) Data Distribution Service, (4) Component Library, (5) Simulation Management, and (6) Scenario Generation. The web-based frontend and backend enable the user to interact with the testbed for tasks such as composing a simulation, running a simulation and retrieving output data. The Testbed Builder application launched from the web frontend is a graphical user interface for the user to drag-and-drop and connect predefined blocks for composing a simulation/scenario generation task. The Builder writes a set of instructions for Simulation Management based on the links between the blocks and the block properties such as the component (executable) associated with a particular block. Management of the distributed simulation is accomplished by Execution and Component Managers. Execution Manager interprets the instructions provided by the Builder to instruct the Component Managers to download components from the Component Library to specified computers and to start them up. Once started, the components communicate with each other by publishing messages and subscribing to messages that are delivered by the Data Distribution Service. The Scenario Generation capability can be used for creating traffic scenarios for Multi-Aircraft Control System, which has been used extensively at NASA for human-in-the-loop-based concept evaluations. The presentation will provide a testbed enabled example scenario of Multi-Aircraft Control System based simulation in which the urban air mobility pilot using the conflict detection and resolution system would interact with the air traffic controllers for resolving conflicts with other aircraft during terminal area operations.

Simulation↗

Evaluating Delta-V Dispersions Using Linear Covariance Techniques with Applications to Rendezvous and Docking

One of the attractive advantages of using linear covariance analysis (LinCov) is that it can accurately produce both navigation errors and trajectory dispersions in a fraction of the time comparable to the standard Monte Carlo analysis approach that is traditionally adopted. It also has the capability of generating delta-v dispersions which is perhaps the most influential performance metric since many aspects of a mission design revolve around the anticipated propellant usage. Accurately capturing this key performance criteria using linear covariance techniques has several subtle caveats raising reasonable doubts and a cautious sense of skepticism. This paper outlines multiple approaches of modeling delta-v usage in LinCov, highlights their limitations and advantages, and ultimately compares their corresponding results to the actual Monte Carlo performance. The theory is also extended to reliably account for an arbitrary thruster configuration and the resultant propellant consumptions due to both translational burns and attitude maneuvers. This provides near instant insight into the impacts of thruster layout design and different vehicle mass properties. These derived techniques are applied and verified using an NRHO rendezvous and docking scenario consistent with upcoming NASA Artemis missions for Orion.

Linear Covariance Analysis↗