Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “SCORE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Classifying thermodynamic cloud phase using machine learning models

Vertically resolved thermodynamic cloud-phase classifications are essential for studies of atmospheric cloud and precipitation processes. The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Thermodynamic Cloud Phase (THERMOCLDPHASE) value-added product (VAP) uses a multi-sensor approach to classify the thermodynamic cloud phase by combining lidar backscatter and depolarization, radar reflectivity, Doppler velocity, spectral width, microwave-radiometer-derived liquid water path, and radiosonde temperature measurements. The measured pixels are classified as ice, snow, mixed phase, liquid (cloud water), drizzle, rain, and liq_driz (liquid+drizzle). We use this product as the ground truth to train three machine learning (ML) models to predict the thermodynamic cloud phase from multi-sensor remote sensing measurements taken at the ARM North Slope of Alaska (NSA) observatory: a random forest (RF), a multi-layer perceptron (MLP), and a convolutional neural network (CNN) with a U-Net architecture. Evaluations against the outputs of the THERMOCLDPHASE VAP with 1 year of data show that the CNN outperforms the other two models, achieving the highest test accuracy, F1 score, and mean intersection over union (IOU). Analysis of ML confidence scores shows that ice, rain, and snow have higher confidence scores, followed by liquid, while mixed, drizzle, and liq_driz have lower scores. Feature importance analysis reveals that the mean Doppler velocity and vertically resolved temperature are the most influential data streams for ML thermodynamic cloud-phase predictions. Lidar measurements exhibit lower feature importance due to rapid signal attenuation caused by the frequent presence of persistent low-level clouds at the NSA site. The ML models' generalization capacity is further evaluated by applying them at another Arctic ARM site in Norway using data taken during the ARM Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) field campaign. The models demonstrated similar performance to that observed at the NSA site. Finally, we evaluate the ML models' response to simulated instrument outages and signal degradation and show that a CNN U-Net model trained with input channel dropouts performs better when input fields are missing.

ARM Aerial Facility↗

Thermodynamic Cloud Phase Classifications Using Machine Learning at NSA and ANX

Vertically resolved thermodynamic cloud phase classifications are essential for studies of atmospheric cloud and precipitation processes. The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) THERMOCLDPHASE Value-Added Product (VAP) uses a multi-sensor approach to classify thermodynamic cloud phase by combining lidar backscatter and depolarization, radar reflectivity, Doppler velocity, spectral width, microwave radiometer-derived liquid water path, and radiosonde temperature measurements. The measured voxels are classified as ice, snow, mixed-phase, liquid (cloud water), drizzle, rain, and liq_driz (liquid+drizzle). We use this product as the ground truth to train three machine learning (ML) models to predict the thermodynamic cloud phase from multi-sensor remote sensing measurements taken at the ARM North Slope of Alaska (NSA) observatory: a random forest (RF), a multilayer perceptron (MLP), and a convolutional neural network (CNN) with a U-Net architecture. Evaluations against the outputs of the THERMOCLDPHASE VAP with one year of data show that the CNN outperforms the other two models, achieving the highest test accuracy, F1-score, and mean Intersection over Union (IOU). Analysis of ML confidence scores shows ice, rain, and snow have higher confidence scores, followed by liquid, while mixed, drizzle, and liq_driz have lower scores. Feature importance analysis reveals that the mean Doppler velocity and vertically resolved temperature are the most influential datastreams for ML thermodynamic cloud phase predictions. The ML models’ generalization capacity is further evaluated by applying them at another Arctic ARM site in Norway using data taken during the ARM Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) field campaign. Finally, we evaluate the ML models’ response to simulated instrument outages and signal degradation.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of high-contact-ratio spur gears with profile modification

Scoring tests, surface fatigue tests, and single-tooth bending fatigue tests were conducted with four sets of spur gears of standard design and three sets of spur gears of the new tooth form (NTF) design. Scoring tests were conducted in a gear test rig at a speed of 10,000 rpm using a synthetic polyester C5, C6, C7 substituted pentaerythritol oil. Surface fatigue tests were conducted in the same rig at a speed of 10,000 rpm and Hertz stresses of 173 x 10 to the 7th power and 143 x 10 to the 7th power Pa (250,000 and 214,000 psi). Single tooth bending fatigue tests were conducted on both the standard and NTF gears at an initial load that produced a 10.35 x 10 to the eight power Pa (150,000-psi) bending stress. The gears were load cycled to failure or for 3 x 10 to the 6th power cycles, whichever occurred first. The load was increased after test until failure occurred at 3 x 10 to the 6th power cycles or less. Both the standard and NTF gears scored at a gear bulk temperature of approximately 409 K (277 F). At this temperature the load on the NTF gears was 22 percent less than the load on the standard gears. The scoring failure was a function of gear bulk temperature, where for a given lubricant the temperature is a function of gear design, operating load, and speed.

Townsend, D. P.↗

Are Soccer Matches Badly Designed Experiments?

Models in which the number of goals scored by a team in a soccer match follow a Poisson distribution, or a closely related one. have been widely discussed. We here consider a soccer match as an experiment to assess which of two teams is superior and examine the probability that the outcome of the experiment (match) truly represents the relative abilities of the two teams. Given a final score it is possible by using a Bayesian approach to quantify the probability that it was or was not the case that 'the best team won'. For typical scores, the probability of a misleading result is significant. Modifying the rules of the game to increase the typical number of goals scored would improve the situation, but a level of confidence that would normally be regarded as satisfactory could not be obtained unless the character of the game were radically changed.

Skinner, G. K.↗

Soccer Matches as Experiments - How Often Does the 'Best' Team Win?

Models in which the number of goals scored by a team in a soccer match follow a Poisson distribution or a closely related one, have been widely discussed. We here consider a soccer match as an experiment to assess which of two teams is superior and examine the probability that the outcome of the experiment (match) truly represents the relative abilities of the two teams. Given a final score it is possible by using a Bayesian approach to quantify the probability that it was or was not the case that the best team won. For typical scores, the probability of a misleading result is significant. Modifying the rules of the game to increase thc typical number of goals scored would improve the situation, but a level of confidence that would normally be regarded as satisfactory could not be obtained unless the character of the game were radically changed.

Skinner, Gerald K.↗

Rapid Corner Detection Using FPGAs

In order to perform precision landings for space missions, a control system must be accurate to within ten meters. Feature detection applied against images taken during descent and correlated against the provided base image is computationally expensive and requires tens of seconds of processing time to do just one image while the goal is to process multiple images per second. To solve this problem, this algorithm takes that processing load from the central processing unit (CPU) and gives it to a reconfigurable field programmable gate array (FPGA), which is able to compute data in parallel at very high clock speeds. The workload of the processor then becomes simpler; to read an image from a camera, it is transferred into the FPGA, and the results are read back from the FPGA. The Harris Corner Detector uses the determinant and trace to find a corner score, with each step of the computation occurring on independent clock cycles. Essentially, the image is converted into an x and y derivative map. Once three lines of pixel information have been queued up, valid pixel derivatives are clocked into the product and averaging phase of the pipeline. Each x and y derivative is squared against itself, as well as the product of the ix and iy derivative, and each value is stored in a WxN size buffer, where W represents the size of the integration window and N is the width of the image. In this particular case, a window size of 5 was chosen, and the image is 640 480. Over a WxN size window, an equidistance Gaussian is applied (to bring out the stronger corners), and then each value in the entire window is summed and stored. The required components of the equation are in place, and it is just a matter of taking the determinant and trace. It should be noted that the trace is being weighted by a constant k, a value that is found empirically to be within 0.04 to 0.15 (and in this implementation is 0.05). The constant k determines the number of corners available to be compared against a threshold sigma to mark a valid corner. After a fixed delay from when the first pixel is clocked in (to fill the pipeline), a score is achieved after each successive clock. This score corresponds with an (x,y) location within the image. If the score is higher than the predetermined threshold sigma, then a flag is set high and the location is recorded.

Morfopoulos, Arin C.↗

Smart Image Enhancement Process

Contrast and lightness measures are used to first classify the image as being one of non-turbid and turbid. If turbid, the original image is enhanced to generate a first enhanced image. If non-turbid, the original image is classified in terms of a merged contrast/lightness score based on the contrast and lightness measures. The non-turbid image is enhanced to generate a second enhanced image when a poor contrast/lightness score is associated therewith. When the second enhanced image has a poor contrast/lightness score associated therewith, this image is enhanced to generate a third enhanced image. A sharpness measure is computed for one image that is selected from (i) the non-turbid image, (ii) the first enhanced image, (iii) the second enhanced image when a good contrast/lightness score is associated therewith, and (iv) the third enhanced image. If the selected image is not-sharp, it is sharpened to generate a sharpened image. The final image is selected from the selected image and the sharpened image.

Jobson, Daniel J.↗

Automated Grouping of Opportunity Rover Alpha Particle X-Ray Spectrometer Compositional Data

The Alpha Particle X-ray Spectrometer (APXS) conducts high-precision in situ measurements of rocks and soils on both active NASA Mars rovers. Since 2004 the rover Opportunity has acquired around 440 unique APXS measurements, including a wide variety of compositions, during its 42+ kilometers traverse across several geological formations. Here we discuss an analytical comparison algorithm providing a means to cluster samples due to compositional similarity and the resulting automated classification scheme. Due to the inherent variance of elements in the APXS data set, each element has an associated weight that is inversely proportional to the variance. Thus, the more consistent the abundance of an element in the data set, the more it contributes to the classification. All 16 elements standard to the APXS data set are considered. Careful attention is also given to the errors associated with the composition measured by the APXS - larger uncertainties reduce the weighting of the element accordingly. The comparison of two targets, i and j, generates a similarity score, S(sub ij). This score is immediately comparable to an average ratio across all elements if one assumes standard weighted uncertainty. The algorithm facilitates the classification of APXS targets by chemistry alone - independent of target appearance and geological context which can be added later as a consistency check. For the N targets considered, a N by N hollow matrix, S, is generated where S = S(sup T). The average relation score, S(sub av), for target N(sub i) is simply the average of column i of S. A large S(sub av) is indicative of a unique sample. In such an instance any targets with a low comparison score can be classified alike. The threshold between classes requires careful consideration. Applying the algorithm to recent Marathon Valley targets indicates similarities with Burns formation and average-Mars-like rocks encountered earlier at Endeavour Crater as well as a new class of felsic rocks.

VanBommel, S. J.↗

Monitoring Airspace Complexity and Determining Contributing Factors

The national airspace has evolved over many years to accommodate increased traffic demand [1] while simultaneously maintaining one of the safest forms of transportation [2], [3]. One of the reasons for this success is the ability of the system and the operators to adapt and accommodate to situations that routinely disrupt optimal operations. These situations may include: adverse weather, delays, early arrivals, equipment outages, and other factors that are outside the operators ability to control. These factors can lead to states where automation is unable to properly handle these issues and therefore air traffic controllers and pilots have to intervene, ultimately increasing communication between operators resulting in higher workload. As controller workload increases to handle sub-optimal operating conditions this can be viewed as an increase in complexity. The reasoning for this is because humans are now required to make tactical decisions in response to external factors, resulting in a departure from the strategic plan where operations would be more efficiently managed. Human operators control airspace complexity under rigid regulations that are constantly changing. The airspace is divided into sectors and the number of aircraft assigned to each controller is limited for safe handling. There has been past work that devised airspace complexity metrics in commercial aviation and related these metrics to controller workload (e.g., [4],[5]). The upper bounds on the system load are pre-determined. Such bounds on complexity make for a safe system, but the system cannot scale and adapt to autonomous, dense, and heterogeneous traffic, including the many types of Unmanned Aerial Vehicles (UAVs) envisioned to be added to the operations. We hypothesize that, as traffic density and heterogeneity grow, and other key metrics change, there will be phase transitions at which the way traffic should be managed changes significantly [6]. We offer a method for in-time detection of contributing factors that lead to phase transitions, characterized by increased complexity. To the best of our knowledge, there is no tool similar to our proposed effort that identifies such contributing factors or precursor patterns. To define the scope we are proposing to measure complexity from the viewpoint of the Terminal Radar Approach Control Facilities (TRACON) controller’s perspective. In particular we are analyzing arrivals into KSFO. With safety as the top concern for airspace operators, it is important to recognize that as density and heterogeneity grow, the focus of the system will change. Times of the day when the airspace has low density and heterogeneity, the flights will follow more efficient paths where the aircraft move on established routes that are more or less directly to the destination. However, when density and heterogeneity increases, the system will begin changing focus to avoiding conflicts and collisions and route the flights in a more flexible way. Higher flexibility requires more communication and coordination between controllers and pilots which the current automation is unable to handle. This paper proposes a novel approach that monitors airspace complexity at multiple scales, uses a Machine Learning-based tool that predicts when operations will transition to a regime of greater complexity, and identifies actions that can reduce the complexity while still maintaining efficient and safe operations. We demonstrate our proposed approach using data from multiple complementary sources. This includes, but is not limited to: historical aircraft surveillance data from NASA’s Sherlock Data Warehouse [7], METAR weather data, and airport configuration data from Aviation System Performance Metrics (ASPM). The surveillance data flight paths are sampled at a variable sample rate — increasing as the aircraft approaches the airport. This is due to how Sherlock manages flight track stitching between different radar facilities which have different sampling rates. The weather and performance data are logged at defined intervals throughout the day at a courser refresh rate. In addition to the logged data and metrics, we leverage pre-defined Standard Terminal Arrival Routes (STARs) procedures to characterize the path of each flight. Each flight files for one of these routes in the flight plan well before entering the terminal airspace, and approximately follows the route until it leaves the STAR, typically on the final fix of a runway transition. However, most flights do not always fly the full STAR procedure to completion [8], but the majority do adhere to the fixes within the common route of the procedure. Our approach leverages fixes in the common route of each of the STARs to build a reference path to the airport. This allows us to characterize the flight paths in what we are defining as the “maneuvering area” (the airspace between the STAR and before the flight is lined up on the runway’s final approach) to determine how off nominal the flights are to calculate its complexity score. Determining airspace complexity is a concept that does not have a concrete answer. In designing this metric, we consider what increases the workload for the air traffic controllers. Consequently more specialized vectoring maneuvers results in higher workload. Accordingly, we start with a theory: each flight has a direct path it takes from the STAR’s common route to the final approach’s outer marker fix for the flight’s landing runway. It is important to note that the direct path is only used as a reference. If the majority of the flights have a large consistent offset as compared to other routes it does not necessarily mean that those flights have higher complexity. We are merely building a distribution based on this direct path for that particular STAR and runway pair to determine the normal mode of operations for that route. Flights that are in the upper tail of these distributions will result in higher complexity scores and flights that fly in the median will represent the normal mode of operations and therefore will have lower complexity scores. Since flights following each STAR route take different paths to the airport, we have a different distribution for each STAR route and therefore can model these distributions to compute a complexity score from their respective normalized distributions. To evaluate the effectiveness of our proposed airspace complexity metric we will compare against an established approach based on trajectory clustering [9]. This unsupervised learning technique consists of the following steps: (1) identify the general maneuvering areas (waypoints) by performing $\kappa$-means or DBSCAN clustering on locations where aircraft frequently turn based on the surveillance radar track data, (2) map flight trajectories onto sequences of waypoints, and (3) cluster the sequences based on their common subsequences. From a high-level perspective, this baseline model learns nominal operations in the airspace through the sequence of waypoints that are representative of where aircraft change direction and defines deviations from the nominal operations as “complex.” Therefore, more deviations from the nominal operations correspond to higher complexity values. For our validation, we re-implemented this technique and tune model hyper-parameters to correctly detect waypoints for the arrival traffic into the San Francisco bay area. We will compute the complexity measure over a one-year period using our proposed technique as well as the baseline. Our validation will be based on each technique’s ability to detect a set of undesirable outcomes (e.g., go-arounds, holding patterns, average time in the airspace, etc.). Since our current complexity metric is derived from the offset from the direct reference path, it’s important to understand what causes these offsets. In many of the flights with high offset distance, flights performing holding patterns and S turns can be observed. These maneuvering tactics are utilized to add distance between the aircraft and the destination runway to prevent multiple flights from having conflicting arrival times. In order to predict a rise in complexity (or the precursor to complexity), it’s necessary to be able to identify these potential conflicts (which in turn, result in higher offsets). To do this, we define a “representative flight” for each STAR route and runway pair. This flight is approximately the path the flight would take if there was a clear path with no other flights in the airspace — including the time remaining to the airport. We first identify the flights for a given STAR runway pair using the offset to the reference path distributions that fall between the 44-55 percentiles. This yields the flights that conform to the most normal mode of operation. Each of these flights is partitioned based on the percent complete from the entry point into the maneuvering areas from 0\% – 100\% complete. Then for each percent “bin”, we take the median value of the flight’s latitude/longitude coordinates, airspeed, and (non causal) time remaining to the airport to construct a lookup table for each percent complete bin on a given route. As a flight enters the maneuvering area, we can find the estimated arrival time of a flight to the airport by finding the closest point to the representative path’s percent complete bin (relative to the flight’s current position at any snapshot in the airspace) and therefore retrieve the corresponding remaining time left on the “representative path”. We assume that the flight will follow the representative path to completion when deriving these estimates. We can then compare these estimated arrival times against other flights for the same snapshot in time to identify potential conflicts. If more flights are estimated to arrive within a tolerance window than there are runways available, then we have a potential conflict. We can use this derived measure along with other factors expected to add disruption to the operation such as weather and runway configuration changes as an input to machine learning tools to detect precursors that increases in our complexity measure. This novel method will assist in uncovering insights into the contributing factors that lead to increased complexity that may allow for in-time responses to avoid reaching a high complexity state in the airspace.

complexity↗

Selection of Next Priority IMPACT Medical Conditions Based on Available Terrestrial and Spaceflight Data

BACKGROUND: As the era of exploration class missions begins, identification of medical conditions that may occur and require management becomes essential for the modeling of medical risk. To this end, NASA has developed IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces), a suite of tools to assist in assessment of medical risk analysis. It has incorporated an initial list of the 120 conditions of highest concern, labeled the IMPACT condition list 1.0 (ICL 1.0). This abstract describes a method for prioritizing the 92 conditions included on the Proposed Future Conditions List (PFCL) for inclusion in future iterations of the ICL. OVERVIEW: To construct the Prioritized Proposed Future Conditions List (P-PFCL), each condition on the PFCL was scored as “low,” “medium,” or “high” on each of four variables: incidence, likelihood of significant task impairment, diagnostic and treatment complexity, and treatment futility. Qualitative assessment using clinical judgement was utilized to score complexity, futility, and likelihood of impairment. Incidence was assessed quantitatively using spaceflight data and/or analog populations where available then assigned a score using established cutoffs. Logarithmic numerical values were assigned to each category label. A Prioritization Score was generated for each condition by taking the product of incidence and likelihood of task impairment (risk) divided by the product of complexity and futility (difficulty of care), with higher values corresponding to higher priority for future inclusion in the ICL. DISCUSSION: The described methods allow for the generation of a ranked P-PFCL to act as a decision support tool for selection of the next generation of modeled medical conditions. Some of the conditions ranked highly on the P-PFCL include EVA-related upper and lower extremity sprain/strain, iron deficiency, delirium, and hypertension, among others. While this effort does not attempt to quantify the absolute risk associated with each condition, it does attempt to semi-quantitatively estimate the risk of each condition relative to the other possible conditions. This tool in concert with subject matter expert opinion could optimize the future use of limited resources thereby producing a more accurate medical risk model, which will be essential to the upcoming exploration class missions.

Michael Pohlen↗

Modeling and Simulation Credibility Assessment of Finite Element Models for Fall From Heights Injury Scenarios

Finite element (FE) computational models of the whole human body are being used to investigate the injury modalities and mechanisms arising from suited astronauts’ ExtraVehicular Activities (EVAs). These FE whole body models – simplified Elemance and THUMS pedestrian versions – have undergone a series of verification and validation (V&V) assessments for applications in automotive, domestic, and contact sports arenas. However, the FE models’ Modeling and Simulation (M&S) credibility assessments for simulating suited astronaut injuries due to fall from heights in altered gravity environments are limited. The current study evaluates the M&S credibility of simplified Elemance and THUMS pedestrian FE models based on eight credibility factors – data pedigree, input pedigree, code verification, solution verification, conceptual validation, referent validation, results uncertainty, and results robustness – as per NASA-STD-7009A [1]. These credibility factors were assigned ordinal scores ranging between 0 to 4, with a score of 0 in the case of insufficient evidence and 4 when all data for the model is known, used appropriately, and traceable to the Real-World System (RWS) with acceptable accuracy, precision, and uncertainty. The credibility factor sufficiency thresholds were assigned upon guidance from NASA EVA subject matter experts. The factor scores were ascertained through the evaluation of M&S credibility assessment procedures described in the published literature and FE model reports. The M&S credibility scores for the Elemance and THUMS FE models vary between 1 and 3 for data pedigree, input pedigree, and referent validation credibility factors, and between 0 and 1 for code verification, solution verification, conceptual validation, results uncertainty and results robustness. The results show that these FE models are credible for simulating fall from heights injury scenarios in specific kinematic and kinetic ranges that correlate with highway and contact sports-related external conditions. Multiple credibility elevation strategies are prescribed to improve the Elemance and THUMS models’ credibility for use in the EVA fall from height injury scenarios.

R A Perkins↗

ExMASS: A viable model for authentic student-scientist research partnerships

The Exploration of the Moon and Asteroids by Secondary Students (ExMASS) program provides students the opportunity to conduct authentic, student-led, open-inquiry research with assistance from their teacher and a scientist. One goal of the program is to enhance participants’ attitudes toward science. Most students enter the ExMASS program with positive attitudes toward science. However, the attitude survey developed for ExMASS measured small but statistically significant increases in attitudes. This paper discusses the ExMASS program, an attitude towards science survey designed for the program’s specific audience, and the results of statistical analyses of the survey. The validity of the attitudes survey was tested with an exploratory factor analysis (EFA). The results of the EFA revealed two salient factors measured by the survey: 1) Personal Importance of Science and 2) Importance of Science to Society. A reliability analysis revealed a Cronbach alpha value of 0.89. Paired t-tests revealed statistically significant differences in students’ attitudes before and after the program. For Factor 1, participants’ before scores were statistically significantly lower than their post scores, t(88) = 3.16, p<0.05. For Factor 2, participants’ before scores were statistically significantly lower than their post scores, t(88) = 4.47, p<0.05. ExMASS has been shown to be a practical model for programs designed to increase high school student attitudes toward science. More information on the ExMASS program is available at https://www.lpi.usra.edu/exploration/education/hsResearch/.

science↗

Batch Effect Correction Methods for NASA GeneLab Transcriptomic Datasets

RNA sequencing (RNA-seq) data from space biology experiments promise to yield invaluable insights into the effects of spaceflight on terrestrial biology. However, sample numbers from each study are low due to limited crew availability, hardware, and space. To increase statistical power, spaceflight RNA-seq datasets from different missions are often aggregated together. However, this can introduce technical variation or "batch effects", often due to differences in sample handling, sample processing, and sequencing platforms. Several computational methods have been developed to correct for technical batch effects, thereby reducing their impact on true biological signals. In this study, we combined 7 mouse liver RNA-seq datasets from NASA GeneLab (part of the NASA Open Science Data Repository) to evaluate several common batch effect correction methods (ComBat and ComBat-seq from the sva R package, and Median Polish, Empirical Bayes, and ANOVA from the MBatch R package). We quantitatively evaluated the ability of these methods to correct for technical batch variables in space biology RNA-seq data using the following criteria: BatchQC, principal component analysis, dispersion separability criterion, log fold change correlation, and differential gene expression analysis. Each batch variable / correction method combination was then assessed using a custom scoring approach to identify the optimal correction method for the combined dataset, by geometrically probing the space of all allowable scoring functions to yield an aggregate volume-based scoring measure. Finally, we describe the way in which the GeneLab multi-study analysis and visualization portal will allow users to examine the presence or absence of batch effects using multiple metrics. If the user chooses to perform batch effect correction, the scoring approach described here can be implemented to identify the optimal correction method to use for their specific combined dataset prior to analysis.

Lauren M. Sanders↗

A Preliminary Study on the Feasibility of Large Language Models for Detecting Micro-Behaviors Among Team Members in Space Missions

Large-language models (LLMs) have been recently used for spoken language understanding (SLU) to infer meaning and semantics from speech in tasks such as speaker intent and sentiment classification. Due to being trained on large amounts of data, and their ability to understand context and relationships between words, LLMs are competent, enabling them to generalize across tasks without requiring many task-specific training samples. This research examines the feasibility of few-shot learning in LLMs for detecting subtle, brief, and possibly unconscious interactions between team members, called ``micro-behaviors," and provides insights into the appropriate design of LLMs for this task. Our data came from 5 teams participating in a 45-day mission at the US National Aeronautics and Space Administration’s (NASA) Human Exploration Research Analog (HERA). More specifically we used data collected from team interaction battery (TIB) tasks teams performed five times in-mission which comprise an average 1.5 hours of conversation data per day. Micro-behaviors were coded according to an adapted version of Smith & Griffins (2022) theoretical framework in terms of Violation (i.e., presence of valenced behavior, uplifting/positive or discouraging/negative), Intensity (i.e., force of behavior in terms of how uplifting or discouraging is the behavior), and Intent (i.e., motive of the behavior in terms of whether it was deliberate or unintentional). We explore the ability of LLMs to detect the presence and intensity of micro-behaviors. We examine employing and fine-tuning readily available LLMs (i.e., RoBERTa, DistilBERT), as well as prompting state-of-the-art sequence classification models (i.e., Llama-2, Llama-3). In a total of 13,058 conversational turns (17.8% uplifting, 3.3% discouraging, 75.76% neutral, 3.14% nulls), we compute the macro F1-score of the 3-way micro-behavior classification task (i.e., classifying among uplifting, discouraging, and neutral; 33% chance). Results indicate that the RoBERTa model achieves a F1-score of 36.2% (uplift: 43.3% precision (P), 15.1% recall (R); discourage: 20% P, 0.5% R). These results significantly improve when we augment the data via paraphrasing in the RoBERTa model, reaching a 41.2% macro F1-score (uplift: 37.7% P, 86.3% R; discourage: 3.5% P, 1.8% R). Finally, the Llama-2 model with 3-shot prompting yields 38% macro F1-score (uplift: 28.7% P, 20% R; discourage: 7.2% P, 18% R), which is slightly better compared to the RoBERTa model without data augmentation, highlighting the effectiveness of sequence classification models in detecting minority classes with a small sample size. Findings indicate that LLMs hold potential to detect subtle behaviors in conversations, which could be valuable in assessing team behavior in space exploration missions. Future studies will evaluate the performance of different LLM prompting strategies or fine-tuning methods.

Ankush Raut↗

Results for the Subjective Habitability Acceptability Questionnaire From HERA C5 and C6

BACKGROUND The impact of habitability on psychological and performance outcomes is recognized as an important factor in habitat and vehicle design—especially as missions increase in length. Formal evaluation of habitability’s impact is possible using the Spaceflight Habitability Acceptability Questionnaire (SHAQ), which was psychometrically validated by Roma et al. in 2022 [1]. The SHAQ survey systematically quantifies the relationships between habitat areas (e.g., bedroom, kitchen, hygiene area, office, etc.) and behavioral health and performance (BHP) outcomes (i.e., individual and team performance, mood, stress, sleep, social interaction), and considers the moderating effects of key habitability aspects (i.e., privacy, control, convenience, efficiency, comfort, social density) on that relationship. METHOD Respondents were HERA participants from Campaign 5 and 6 (n = 32). Data from three environments were used in our SHAQ analyses: SHAQ in HERA (in-mission), SHAQ at Home (pre-mission), and SHAQ in Hotel (pre- and post-mission for C6, just pre-mission for C5). For each habitat area, respondents are asked how that area layout impacts six BHP outcomes on a 201-point visual analog scale with the anchors: impairs, has no effect, and facilitates. They also had the option to respond “Not Applicable”. Follow up questions asked why six key habitability aspects (e.g., privacy) influenced the way the habitat area affected a given outcome. Participants indicated whether a habitat area’s habitability aspects were inadequate (low score) to adequate (midpoint) to ideal (high score), using a 201-point visual analog scale as well but were binned to a 7-point scale for analyses. RESULTS AND CONCLUSION When comparing across environments, ratings of BHP outcomes were worse in HERA compared to home and pre-mission hotel, but rebounded for the post-mission hotel such that post-mission hotel rating were often higher than all previous ratings. Within HERA, ratings of how a habitat area impacted an outcome skewed towards facilitation with communal areas (i.e., galley and office) rated most facilitating. The follow up questions about why six key aspects related to habitability moderated outcomes indicated that most habitat areas were rated as having “adequate” habitability characteristics. Privacy was the only moderator with average scores below adequate for all habitat areas. The office had the most moderators scoring above adequate for all habitat areas. Ratings indicated small to moderate effects of layouts on BHP outcomes: with facilitating layouts averaging between 4 and 5, and impairing layouts averaging between 2 and 4. The sleeping module layout facilitated sleep, with ratings indicating adequate comfort but inadequate privacy. The galley layout facilitated social interaction and mood, with ratings indicating adequate efficiency and convenience. The office layout facilitated social interaction, team and individual performance, with ratings similarly indicating adequate efficiency and convenience. The hygiene module worsened individual performance, mood and stress, with ratings indicating less than adequate privacy and social density. Our findings suggest there is value in having dedicated team and individual spaces. REFERENCES [1] Roma, P. G., Landon, L.B., Spencer, C. A., Whitmire, A. M., & Williams, T. J. (2022). SHAQ: Development and Validation. Human Factors: Special Issue on Spaceflight, 65(6), 1074-1104. ACKNOWLEDGEMENTS This project was funded by the NASA Human Research Program directed task Human Factors and Behavioral Performance in HERA (PI = S. Bell) to the BHP Lab. KBR Inc. and JES Tech authors were supported by KBR’s Human Health and Performance Contract NNJ15HK11B with NASA.

behavioral health↗

Digitalization mapping and assessment process supporting ION strategic transformation activities

The existing fleet of commercial nuclear power plants (NPPs) are an important asset in the nation’s portfolio of electrical generating resources. Their continued safe and reliable operation are critical to providing a large source of carbon-free electricity to power the nation’s economy. The United States Department of Energy’s (DOE) Light Water Reactor Sustainability (LWRS) Program develops the scientific bases, methods, and tools, for the continued safe and economical operation of the nation's commercial NPPs. The Plant Modernization Pathway within LWRS Program focuses on providing guidance to industry on the full-scale implementation of modernization solutions for NPPs that significantly reduce the technical and financial risks associated with modernization. This research is focused on helping the nuclear industry understand how to digitize and digitalize their NPPs so that they can design their modernization solutions to be scalable, sustainable, and integrated both laterally and horizontally within their organization. That is, this research creates a digital transformation in NPPs by reshaping cultural mindsets and by identifying business efficiencies. In partnership with industry, and using four previously established guiding principles for digitalization, this research supported NPP modernization through assessing readiness for digitalization as a means to achieve integrated operations for nuclear. Specifically, this research created an assessment to review an entire organization’s work processes to gather information about the digitalization health of the plant. The assessment tools were administered to plant employees, and the results were used to develop a digitalization plan. The survey assessment identified the optimal candidate processes that would most benefit from a digitalization initiative which were revealed through analytical frameworks. One analysis calculated mean digitalization health indicator scores for all endorsed activities which allowed the researchers to rank and color code the results for easy identification. Individual health indicator scores are also provided, should our industry partner wish to understand these findings according to their own organizational priorities, business considerations and desired end-state. The results were also analyzed from the perspective that organizations are comprised of different types of innovators (e.g., generators, optimizers, conceptualizers, and implementers), which differentially affects the organization’s ability to comprehend and adapt to change (i.e., opportunities to innovate). Understanding the relative composition of innovator types at an NPP allows them to gather insights into the strengths and weaknesses they have in innovating how work is performed. For the utility that partnered with this research team, the results showed that implementers make up the largest portion of respondents and conceptualizers the smallest portion. Knowing the proportion of innovator types gave this organization insights on how they can effectively implement their innovation solutions. Additionally, the results were analyzed from a technical, economic, and risk perspective to identify and quantify work reduction opportunities (WROs). Recognizing that not all cost-saving opportunities are the same, a Technical, Economic and Risk Assessment (TERA) was performed to evaluate WROs to identify areas of greatest potential and lowest risk. The key results from TERA included a digitalization opportunity score for each activity, and a calculation of potential cost savings. These two outputs formed the bases for calculating a priority index and rank for the activities/processes assessed. From the prioritization calculations, TERA can then help the utility 1) decide what digitalization priorities to invest money in implementing and then 2) calculates how much should be invested in the digitalization initiatives selected to achieve cost savings and/or an acceptable return on investment. Last, onsite interviews revealed several inefficiencies in the standard work processes that occur cross-departmentally that are due to the absence of digitized and digitalized processes. Examples of these include time spent scanning paper documents and then uploading the documents electronically, obtaining signatures, and searching for desired information. This represents a digital but not digitalized process. Over 15 opportunities to improve work processes were identified through this multi-method digitalization assessment. The various analytical assessments used (e.g., TERA, digitalization health indicator scores), as well as discussions with the utility partner, corroborated that all the opportunities identified had a strong potential to make work processes more efficient and to improve overall performance of the NPP.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Publication and Impact of Preprints Included in the First 100 Editions of the CDC COVID-19 Science Update: Content Analysis

Preprints are publicly available manuscripts posted to various servers that have not been peer reviewed. Although preprints have existed since 1961, they have gained increased popularity during the COVID-19 pandemic due to the need for immediate, relevant information. The aim of this study is to evaluate the publication rate and impact of preprints included in the Centers for Disease Control and Prevention (CDC) COVID-19 Science Update and assess the performance of the COVID-19 Science Update team in selecting impactful preprints. All preprints in the first 100 editions (April 1, 2020, to July 30, 2021) of the Science Update were included in the study. Preprints that were not published were categorized as “unpublished preprints.” Preprints that were subsequently published exist in 2 versions (in a peer-reviewed journal and on the original preprint server), which were analyzed separately and referred to as “peer-reviewed preprint” and “original preprint,” respectively. Time to publish was the time interval between the date on which a preprint was first posted and the date on which it was first available as a peer-reviewed article. Impact was quantified by Altmetric Attention Score and citation count for all available manuscripts on August 6, 2021. Preprints were analyzed by publication status, publication rate, preprint server, and time to publication. Of the 275 preprints included in the CDC COVID-19 Science Update during the study period, most came from three servers: medRxiv (n=201, 73.1%), bioRxiv (n=41, 14.9%), and SSRN (n=25, 9.1%), with 8 (2.9%) coming from other sources. Additionally, 152 (55.3%) were eventually published. The median time to publish was 2.3 (IQR 1.4-3.7). When preprints posted in the last 2.3 months were excluded (to account for the time to publish), the publication rate was 67.8%. Moreover, 76 journals published at least one preprint from the CDC COVID-19 Science Update, and 18 journals published at least three. The median Altmetric Attention Score for unpublished preprints (n=123, 44.7%) was 146 (IQR 22-552) with a median citation count of 2 (IQR 0-8); for original preprints (n=152, 55.2%), these values were 212 (IQR 22-1164) and 14 (IQR 2-40), respectively; for peer-review preprints, these values were 265 (IQR 29-1896) and 19 (IQR 3-101), respectively. Prior studies of COVID-19 preprints found publication rates between 5.4% and 21.1%. Preprints included in the CDC COVID-19 Science Update were published at a higher rate than overall COVID-19 preprints, and those that were ultimately published were published within months and received higher attention scores than unpublished preprints. These findings indicate that the Science Update process for selecting preprints had a high fidelity in terms of their likelihood to be published and their impact. The incorporation of high-quality preprints into the CDC COVID-19 Science Update improves this activity’s capacity to inform meaningful public health decision-making.

59 BASIC BIOLOGICAL SCIENCES↗

Discrete ordinates analysis of the forced-flight variance reduction technique in Monte Carlo neutral particle transport simulations

This paper presents mathematical formulations and methods to predict the effect of forced-flight variance reduction on Monte Carlo tally variance and calculation time. This includes deducing biasing operators that are then used to construct a history-score probability density function (HSPDF), which represents all possible Monte Carlo random walks and gives the probability of a Monte Carlo history scoring in a tally from a particular phase-space position. The history-score moment equations (HSMEs), the statistical moments of the HSPDF, are then derived to calculate the statistical behavior of the Monte Carlo tally when forced-flight variance reduction is applied. In addition, the future-time equation (FTE) is derived to predict the Monte Carlo computational time as a result of applying forced-flight variance reduction. The solutions of the HSMEs and FTE can be used to predict Monte Carlo computational cost. Furthermore, this work also describes a discrete ordinates method to solve the forced-flight HSMEs and FTE. Several 1-D and 2-D test problems verify that the derivations are performed and implemented correctly.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗