Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “qualitative evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Assessing Mechanical Properties of Spacecraft Materials under Simulated Low Earth Orbit Atomic Oxygen Conditions

During a space mission, spacecraft surface materials are exposed to various damaging environmental factors including high-energy photons, electrons, atomic oxygen (AO) neutrals and ions, micrometeoroids and orbital debris, vacuum, and large temperature fluctuations. The resulting change in spacecraft material properties can significantly impact the performance and durability of spacecraft systems. Even though all aspects of the space environment can lead to the deterioration of spacecraft components, in low Earth orbit (LEO), the threat posed by AO is especially severe in terms of structural and optical damage, particularly to exterior spacecraft components that are susceptible to oxidation. A comprehensive understanding of material AO-induced weathering is essential for mission planning in the LEO environment. The presented work aims to evaluate the alterations in mechanical properties of selected innovative spacecraft materials and surface electronic system designs, such as Kapton® CR film coated front and back-side with polyimide coating containing AO-resistant filler, Kevlar EXO, and Bendable Electrodynamic Dust Shield (BEDS) architecture, under simulated AO exposure, utilizing the photoelasticity phenomenon in which birefringence is induced in a material when it is subjected to mechanical stress. Changes in stress patterns during the deformation of AO-exposed polymers were assessed using a large-field polariscope under varied deformation types. Patterns of colors were used for qualitative evaluations of residual stress. The stress patterns of the AO-exposed polymers were compared to those of the unexposed ones. Also, these patterns were correlated with data from Bidirectional Reflectance Distribution Function (BRDF) and surface morphology studies.

Yuliya Kuznetsova↗

Identifying and Analyzing Uncertainty Structures in the TRMM Microwave Imager Precipitation Product over Tropical Ocean Basins

Despite continuous improvements in microwave sensors and retrieval algorithms, our understanding of precipitation uncertainty is quite limited, due primarily to inconsistent findings in studies that compare satellite estimates to in situ observations over different parts of the world. This study seeks to characterize the temporal and spatial properties of uncertainty in the Tropical Rainfall Measuring Mission Microwave Imager surface rainfall product over tropical ocean basins. Two uncertainty analysis frameworks are introduced to qualitatively evaluate the properties of uncertainty under a hierarchy of spatiotemporal data resolutions. The first framework (i.e. 'climate method') demonstrates that, apart from random errors and regionally dependent biases, a large component of the overall precipitation uncertainty is manifested in cyclical patterns that are closely related to large-scale atmospheric modes of variability. By estimating the magnitudes of major uncertainty sources independently, the climate method is able to explain 45-88% of the monthly uncertainty variability. The percentage is largely resolution dependent (with the lowest percentage explained associated with a 1 deg x 1 deg spatial/1 month temporal resolution, and highest associated with a 3 deg x 3 deg spatial/3 month temporal resolution). The second framework (i.e. 'weather method') explains regional mean precipitation uncertainty as a summation of uncertainties associated with individual precipitation systems. By further assuming that self-similar recurring precipitation systems yield qualitatively comparable precipitation uncertainties, the weather method can consistently resolve about 50 % of the daily uncertainty variability, with only limited dependence on the regions of interest.

Climate↗

A Flight Replanning Tool for Terminal Area Urban Air Mobility Operations

In this presentation we describe and evaluate a flight replanning tool, called the trial planner, for terminal area air transport applications. The trial planner employs predefined airspace structures to generate rerouting options between vertiports. In a simplified definition, vertiports are facilities that provide services for managing the take-off and landing of autonomous or manned electric vertical take-off or landing (eVTOL) aircraft. The airspace structures involved in our test cases are arrival routes that consist of predefined entry points to which a transitional path is computed dynamically from the current position of the aircraft. Rerouting options include the transitional path and approach segment along the arrival route; these rerouting options are not vetted for potential conflicts with other operations until a human operator commits to a choice and forwards the flight plan modification for approval from an automated airspace management service. A selected rerouting option is executed by loading a flight plan file to the aircraft ground control station. We conducted a qualitative evaluation of the trial planner using ratings provided by flight crews and air transportation human factors experts with relevant experience from air traffic control. Evaluation criteria included trust in the route recommendations and adequate explanation for the route options. We detail the logic and implementation of the trial planner, as well as report the results of the evaluation of the implementation herein.

Trial planning↗

Mars Entry, Descent, Landing, and Ascent Systems Sensitivities to Landing Site and Atmospheric Dust

Plans for human missions to Mars continue to go through several architectural changes, dating all the way back to the 1950s [1]. The continuous study, reformulation, and refinement of Mars architectures and system concepts is necessary in order to incorporate evolving mission objectives, technology advancements, and growth in the body of knowledge regarding human factors and the various environments of human space travel. This evolution has continued into the 21st century, with architectures concepts such as NASA’s Design Reference Architecture 5.0 in 2009 [2], The Evolvable Mars Campaign in 2016 [3], and as recently as 2020, an architecture focused on a crewed mission as early as the 2030s that aims to minimizing crewed duration and infrastructure investment for the first mission [4]. Within these architectures, numerous studies around the current concept designs for Mars entry, descent, landing, and ascent (EDLA) systems have been performed over the last half a decade [[5], [6], [7], [8], [9], [10]]. Despite the breadth and depth of these studies, landing site, a key design parameter relevant to the design of EDLA systems, has remained nebulous over the years, largely due to the ever evolving mission objectives and architecture concept over the decades. However, the specific landing site has direct impact on the altitude and atmospheric conditions, which subsequently impact the design of EDLA systems. To accommodate the lack of a specific landing site, a baseline reference altitude of 0 km relative to the Mars Orbiter Laser Altimeter (MOLA), which is similar to an Earth sea level reference, has typically been selected and fixed for these past studies. Similarly, a baseline reference atmosphere has typically been utilized in these studies, either the Mars Global Reference Atmospheric Model [11], or a general 1982 standard warm, high pressure atmosphere model derived from Viking lander data [12]. Fig. 1 shows the range of elevations across the surface of Mars. Current architectures are focused on latitudes greater than 30 degrees north in an effort to ensure access to frozen water ice. From the figure, it is quickly evident that elevations over the range of longitudes at or above this latitude are significantly varied between roughly -4 to +4 km MOLA. Based on these observations, initial qualitative assessments of the impacts of landing site elevation on EDLA systems were performed. Fig. 2 depicts the expected trends in EDLA system mass for variations in both land site latitude and elevation. The background coloring is a qualitative evaluation of the current knowledge on ice water availability at the latitudes. Further modeling and simulation was performed to obtain numerical predictions of sensitivities to these parameters. Results indicate up to +2% to -4% mass variation from the current baseline Mars Ascent Vehicle concept, with the potential for greater than 6% mass variation at latitudes greater than 70 degrees north. However, variations in the descent system due to landing site were much more significant with mass variations in the range of -15% to +30% around the current baseline Mars Descent System concept. Additionally, Fig. 3 provides a depiction of potential Mars atmospheric density variations with both dust and Martian season. Though the Martian atmosphere is relatively thin compared to Earth’s atmosphere, it still poses significant impact on the design of EDLA systems. Variations in atmospheric density indicated by this slice of data would have significant impact on the design of EDLA systems. However, due to the highly complex nature of atmospheric flight, a qualitative assessment could not be performed. Rather, sensitivity results relied on modeling and simulation to provide numerical results for sensitivities on the current EDLA design concepts under consideration. Understanding these sensitivities is vital to the overall systems design of a Mars architecture. The resulting mass impacts on EDLA systems, driven by landing site elevation and dust level variations, has rippling impacts throughout the architecture that, ultimately, impact the viability of the architecture. Results showed fairly minor mass impacts to the current Mars Ascent Vehicle baseline configuration, roughly -0.6% to + 0.4% mass variations, while the Mars Descent System say greater variations due to dust, roughly -4% to +6% around the baseline concept.

EDLA↗

Linking Remotely Sensed Aerosol Types to Their Chemical Composition

Aerosol types measured during the Ship-Aircraft Bio-Optical Research (SABOR) experiment are related to GEOS-Chem model chemical composition. The application for this procedure to link model chemical components to aerosol type is desirable for understanding aerosol evolution over time. The Mahalanobis distance (DM) statistic is used to cluster model groupings of five chemical components (organic carbon, black carbon, sea salt, dust and sulfate) in a way analogous to the methods used by Burton et al. [2012] and Russell et al. [2014]. First, model-to-measurement evaluation is performed by collocating vertically resolved aerosol extinction from SABOR High Spectral Resolution LiDAR (HSRL) to the GEOS-Chem nested high-resolution data. Comparisons of modeled-to-measured aerosol extinction are shown to be within 35% +/- 14%. Second, the model chemical components are calculation into five variables to calculate the DM and cluster means and covariances for each HSRL-retrieved aerosol type. The layer variables from the model are aerosol optical depth (AOD) ratios of (i) sea salt and (ii) dust to total AOD, mass ratios of (iii) total carbon (i.e. sum of organic and black carbon) to the sum of total carbon and sulfate (iv) organic carbon to black carbon, and (v) the natural log of the aerosol-to-molecular extinction ratio. Third, the layer variables and at most five out of twenty SABOR flights are used to form the pre-specified clusters for calculating DM and to assign an aerosol type. After determining the pre-specified clusters, model aerosol types are produced for the entire vertically resolved GEOS-Chem nested domain over the United States and the model chemical component distributions relating to each type are recorded. Resulting aerosol types are Dust/Dusty Mix, Maritime, Smoke, Urban and Fresh Smoke (separated into 'dark' and 'light' by a threshold of the organic to black carbon ratio). Model-calculated DM not belonging to a specific type (i.e. not meeting a threshold probability) is termed an outlier and those DM values that can belong to multiple types (i.e. showing weak probability of belonging to a specific cluster) are termed as Overlap. MODIS active fires are overlaid on the model domain to qualitatively evaluate the model-predicted Smoke aerosol types.

Aerosol Types↗

Assessing data change in scientific datasets

Summary Scientific datasets are growing rapidly and becoming critical to next‐generation scientific discoveries. The validity of scientific results relies on the quality of data used and data are often subject to change, for example, due to observation additions, quality assessments, or processing software updates. The effects of data change are not well understood and difficult to predict. Datasets are often repeatedly updated and recomputing derived data products quickly becomes time consuming and resource intensive and may in some cases not even be necessary, thus delaying scientific advance. Despite its importance, there is a lack of systematic approaches for best comparing data versions to quantify the changes, and ad‐hoc or manual processes are commonly used. In this article, we propose a novel hierarchical approach for analyzing data changes, including real‐time (online) and offline analyses. We employ a variety of fast‐to‐compute numerical analyses, graphical data change representations, and more resource‐intensive recomputations of a subset of the data product. We illustrate the application of our approach using three scientific diverse use cases, namely, satellite, cosmological, and x‐ray data. The results show that a variety of data change metrics should be employed to enable a comprehensive representation and qualitative evaluation of data changes.

97 MATHEMATICS AND COMPUTING↗

Local bilinear computation of Jacobi sets

Abstract We propose a novel method for the computation of Jacobi sets in 2D domains. The Jacobi set is a topological descriptor based on Morse theory that captures gradient alignments among multiple scalar fields, which is useful for multi-field visualization. Previous Jacobi set computations use piecewise linear approximations on triangulations that result in discretization artifacts like zig-zag patterns. In this paper, we utilize a local bilinear method to obtain a more precise approximation of Jacobi sets by preserving the topology and improving the geometry. Consequently, zig-zag patterns on edges are avoided, resulting in a smoother Jacobi set representation. Our experiments show a better convergence with increasing resolution compared to the piecewise linear method. We utilize this advantage with an efficient local subdivision scheme. Finally, our approach is evaluated qualitatively and quantitatively in comparison with previous methods for different mesh resolutions and across a number of synthetic and real-world examples.

97 MATHEMATICS AND COMPUTING↗

Explaining word embeddings with perfect fidelity: a case study in predicting research impact

The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.

Coarse-grained models↗

Error-mitigated data-driven circuit learning on noisy quantum hardware

Application-level benchmarks measure how well a quantum device performs meaningful calculations. In the case of parameterized circuit training, the computational task is the preparation of a target quantum state via optimization over a loss landscape. This is complicated by various sources of noise, fixed hardware connectivity, and generative modeling, the choice of target distribution. Gradient-based training has become a useful benchmarking task for noisy intermediate-scale quantum computers because of the additional requirement that the optimization step uses the quantum device to estimate the loss function gradient. In this work, we use gradient-based data-driven circuit learning to qualitatively evaluate the performance of several superconducting platform devices and present results that show how error mitigation can improve the training of quantum circuit Born machines with 28 tunable parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Ronchi shearing interferometry for wavefronts with circular symmetry

Ronchi testing of a focused electromagnetic wave has in the last few years been used extensively at X-ray free-electron laser (FEL) facilities to qualitatively evaluate the wavefront of the beam. It is a quick and straightforward test, is easy to interpret on the fly, and can be used to align phase plates that correct the focus of aberrated beams. In general, a single Ronchigram is not sufficient to gain complete quantitative knowledge of the wavefront. However the compound refractive lenses that are commonly used at X-ray FELs exhibit a strong circular symmetry in their aberration, and this can be exploited. Furthermore, a simple algorithm that uses a single recorded Ronchigram to recover the full wavefront of a nano-focused beam, assuming circular symmetry, is presented, and applied to experimental measurements at the Matter in Extreme Conditions instrument at the Linac Coherent Light Source.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Particle Filter Based Inference Testing

The primary intent of PAR-FIT (Particle Filter based Inference Testing) is to provide hard inductive evidence that a machine learning model is capable and proven for an individual test input. By examining training data used to form the underlying model functional correlation, an estimate of the reliability that a model will make the correct prediction can be made. The Sequential Probability Ratio Test is used to derive a qualitative evaluation for reliability based on hypothesis testing. The PAR-FIT framework achieves this by implementing a particle filter and the sequential probability ratio test algorithms on the machine learning model training data to determine relevancy of new individual test samples to the training dataset. The kernel function evaluates the local proximity and density of training data used to derive a prediction outcome. Particles are used to probabilistically determine which training data to evaluate for proximity. For test samples that are within a close proximity to and surrounded by multiple training data points, the evaluated reliability of the prediction is high. For test samples that are anomalies not represented by the training dataset, in low density data clusters, or are far from existing data points, the evaluated reliability is low as insufficient training evidence exists to suggest the model is capable of making the correct prediction. Sequential Probability Ratio Test is further used to determine when a hypothesis on whether a signal can be rejected or accepted for use. The ratio test collects sequence information from the particle filter to test whether the signal is anomalous or normal via hypothesis testing of the underlying distributions.

Chen, Edward [Idaho National Laboratory (INL), Ida↗

Technology Innovation Pathways for Distributed Wind Balance-of-System Cost Reduction

This exploratory analysis characterizes the balance-of-system (BOS) cost reduction opportunity for small, commercial, medium, and large distributed wind systems. To do this, we used the National Renewable Energy Laboratory’s (NREL’s) Land-based Balance of System Systems Engineering (LandBOSSE) model (Eberle et al. 2019). This model calculates the capital expenditures (CapEx) associated with installation and the system components (e.g., foundation and electrical infrastructure) other than the rotor nacelle assembly and tower. Then, building on the cost reduction potential assessment, we provide a more qualitative evaluation of prospective technology innovation concepts that may enable realization of those cost reductions for distributed wind systems.

17 WIND ENERGY↗

Improvements of Nuclear Data Evaluations for Lead Isotopes in Support of Next Generation Lead-Cooled Fast Systems

The neutron evaluation of the isotopes that comprise natural lead were undertaken as a part of DOE-NEUP Project #19-16739. The goal of the project was to update the neutron cross sections to account for new differential measurements and incorporate the most up-to-date physics. Shortcomings in the lead cross sections was made known by several independent reports. The work performed here repeated the simulation of all the “benchmark” validation systems and concluded that the major issue is the scattering cross sections in the major lead isotopes above 100 keV. Re-evaluation of 206,207,208 Pb included both the resolved resonance region and fast region evaluations of the cross sections. Combined these regions cover energies from thermal to 20 MeV. The most drastic improvement is the resolved resonance region evaluation of 208 Pb which is now extended from 1.0 to 1.5 MeV. Parameterization of these resonances in the R-matrix provides a superior reconstruction of not only the experimental cross section but also the scattering distributions via the Blatt-Biedenharn formalism. Fast region evaluations of the three major isotopes were done to include new inelastic experimental data from the neutron Time-of-Flight facility at CERN. The culmination of all the changes to the cross section is a drastic improvement in the scattering kernel as shown in Rensselaer Polytechnic Institute (RPI) Quasi-Differential scattering measurements and improved prediction of keff for fast integral experiments. Alongside the new cross sections, new nuclear data covariance (uncertainties) have been computed and are included in the evaluation. Little is changed in the magnitude of the uncertainties but the correlations within the covariance display non-trivial changes. The new evaluations of 206,207,208 Pb have been submitted to the National Nuclear Data Center to be included in the ENDF/B-VIII.1 library. While the cross sections have been updated extensively, knowledge and modeling of the cross sections between 1.0 and 3.0 MeV for the isotopes remain a challenge. Most notably is the double differential elastic cross section for 208 Pb and 206 Pb. New quasi-differential measurements for pure samples of these two nuclei would go a long way in reducing compensating errors in the evaluations. Qualitatively, the project has produced a prosperous collaboration between the nuclear data group at RPI with staff scientists at Brookhaven National Laboratory, Naval Nuclear Laboratory, Oak Ridge National Laboratory, Los Alamos National Laboratory, Lawrence Livermore National Laboratory, and Sandia National Laboratories. Quantitatively this is reflected in seven conference presentations, at least one journal submission, and one doctoral thesis.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Improbability of Post-Closure Criticality in Array of Transuranic Waste Containers after Compaction by Salt Creep at Waste Isolation Pilot Plant

Based on the rationale presented, post-closure nuclear criticality is improbable when room closure compacts containers disposing transuranic (TRU) waste emplaced at the Waste Isolation Pilot Plant (WIPP), an operating repository in bedded salt in southeastern New Mexico. As described in the original WIPP certification, a qualitative estimate of the probability of post-closure criticality in TRU waste produced from atomic energy defense activities has been low either because remote-handled TRU waste canisters are neutronically isolated by the bedded salt or because the low fissile mass in an array of contact-handled TRU waste drums cannot be compacted sufficiently by room closure from salt creep. These situations are still valid for the majority of TRU waste that is disposed at WIPP without any disposal constraints, as updated herein. This report also qualitatively evaluates the probability of criticality after disposal of TRU waste in pipe overpack containers (POC) where every POC in a shipment may have the maximum 200 fissile gram equivalent of 239Pu content. The probability of criticality for a disposal room filled with POCs is estimated during four representative phases of repository evolution: (1) a large salt block falls onto POCs in the first 20 years, (2) salt creep closes a disposal room in the first 1000 years without brine seepage and subsequent gas generation, which permits maximum compaction, (3) some brine seepage occurs into the closed room, which initiates consumption of the fiberboard (cellulose) impact absorber in the POC in the second 1000 years, and (4) full brine inundation of a room and consumption of all fiberboard thereafter. Salt-block fall in the first phase does not greatly disrupt three tiers of POCs. The compacted spacing of POCs in the later three repository conditions is calculated through high-fidelity, geomechanical modeling. Criticality evaluation of compacted 200-g 239 Pu spheres at the compacted spacing shows that neither 12-inch nor 6-inch POCs are critical after the first 1000 years, the second 1000 years, or thereafter as the sea of reflector material changes to represent the three repository conditions. Specifically, fiberboard and iron isolates 239 Pu while dry, and brine reduces the reactivity when a room is partially and fully inundated. Because POC behavior bounds behavior of other standard TRU waste containers, post-closure criticality caused by room closure compacting containers is omitted in the performance assessments for the 2019 and 2026 WIPP compliance re-certification applications to the US Environmental Protection Agency.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Assessing turbulence and mixing parameterizations in the gray-zone of multiscale simulations over mountainous terrain during the METEX21 field experiment

Multiscale numerical weather prediction models transition from mesoscale, where turbulence is fully parameterized, to microscale, where the majority of highly energetic scales of turbulence are resolved. The turbulence gray-zone is situated between these two regimes and multiscale models must downscale through these resolutions. Here, we compare three multiscale simulations which vary by the parameterization used for turbulence and mixing within the gray-zone. The three parameterizations analyzed are the Mellor-Yamada Nakanishi and Niino (MYNN) Level 2.5 planetary boundary layer scheme, the TKE-1.5 large eddy simulation (LES) closure scheme, and a recently developed three-dimensional planetary boundary layer scheme based on the Mellor-Yamada model. The simulation domain includes complex (i.e., mountainous) terrain in Nevada that was instrumented with meteorological towers, profiling and scanning lidars, a tethered balloon, and a surface flux tower. Simulations are compared to each other and to observations, with assessment of model skill at predicting wind speed, wind direction and TKE, and qualitative evaluations of transport and dispersion of smoke from controlled releases. This analysis demonstrates that microscale predictions of transport and dispersion can be significantly influenced by the choice of turbulence and mixing parameterization in the terra incognita , particularly over regions of complex terrain and with strong local forcing. This influence may not be apparent in the analysis of model skill, and motivates future field campaigns involving controlled tracer releases and corresponding modeling studies of the turbulence gray-zone.

54 ENVIRONMENTAL SCIENCES↗

Water narratives in local newspapers within the United States

Sustainable use of water resources continues to be a challenge across the globe. This is in part due to the complex set of physical and social behaviors that interact to influence water management from local to global scales. Analyses of water resources have been conducted using a variety of techniques, including qualitative evaluations of media narratives. This study aims to augment these methods by leveraging computational and quantitative techniques from the social sciences focused on text analyses. Specifically, we use natural language processing methods to investigate a large corpus (approx. 1.8M) of newspaper articles spanning approximately 35 years (1982–2017) for insights into human-nature interactions with water. Focusing on local and regional United States publications, our analysis demonstrates important dynamics in water-related dialogue about drinking water and pollution to other critical infrastructures, such as energy, across different parts of the country. Our assessment, which looks at water as a system, also highlights key actors and sentiments surrounding water. Extending these analytical methods could help us further improve our understanding of the complex roles of water in current society that should be considered in emerging activities to mitigate and respond to resource conflicts and climate change.

54 ENVIRONMENTAL SCIENCES↗

Electrothermal follow display apparatus for electroexplosive device testing

A self-balancing bridge for ascertaining the electrothermal and nonlinear behavior of an electroexplosive device is described. A sinusiodal current is passed through the device which provides a signal in the form of a unique Lissajous display. The display can be qualitatively evaluated and abnormal units can be readily detected.

Rosenthal, L. A.↗

Prediction of pump cavitation performance

A method for predicting pump cavitation performance with various liquids, liquid temperatures, and rotative speeds is presented. Use of the method requires that two sets of test data be available for the pump of interest. Good agreement between predicted and experimental results of cavitation performance was obtained for several pumps operated in liquids which exhibit a wide range of properties. Two cavitation parameters which qualitatively evaluate pump cavitation performance are also presented.

Moore, R. D.↗