SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This is an invited talk on autonomy and workload for an AFRL Blue Sky workshop sponsored by the Florida Institute for Human Machine Studies. The presentation reviews various metrics of workload and how to move forward with measuring workload in a human-autonomy teaming environment.
This paper describes a study comparing the use of paper strips with virtual flight strips depicted on a new user interface, the Ramp Traffic Console (RTC), designed for use by ramp controllers to be used in place of paper strips. A Human-In-the-Loop (HITL) experiment was performed as the fifth in a series of six HITL simulation studies designed to evaluate a pushback Decision Support Tool (DST) concept for Charlotte Douglas International Airport (CLT). Workload and usability were assessed in post-run and post-study questionnaires. In the RTC virtual flight strip condition, post-run questionnaire results show lower workload ratings across all aspects of workload; additionally a trend is found toward increased usability ratings. Post-study questionnaire results indicate a preference for RTC over paper strips. Additional research is suggested with more training runs and a greater number of participants to increase statistical power. It is also suggested that this new technology be re-evaluated as a part of the ATD-2 (Airspace Technology Demonstration 2) field testing activities.
Air traffic controllers (ATCOs) must maintain a consistently high level of human performance in order to maintain flight safety and efficiency. In current control environments, performance-influencing factors such as workload, fatigue and situation awareness (SA) can co-occur, and interact, to affect performance. However, multifactor influences and the association with performance are under-researched. This study utilized a high fidelity human in the loop enroute air traffic control simulation to investigate the relationship between workload, situation awareness and ATCO performance. The study aimed to replicate and extend Edwards, Sharples, Wilson and Kirwan's (2012) previous study and confirm multifactor interactions with a participant sample of ex-controllers. The study also aimed to extend Edwards et al.'s previous research by comparing multifactor relationships across 4 automation conditions. Results suggest that workload and SA may interact to produce a cumulative impact on controller performance, although the effect of the interaction on performance may be dependent on the context and amount of automation present. Findings have implications for human-automation teaming in air traffic control, and the potential prediction and support of ATCO performance.
A human-in-the-loop simulation was conducted that examined an approach to adjusting airport arrivals in such a way as to enable higher departure throughput while maintaining arrival throughput. This approach, referred to as Departure-Sensitive Arrival Spacing (DSAS), leverages the capabilities of the Terminal Sequencing and Spacing (TSS) system with an additional Decision Support Tool (DST) to assign more precise arrival spacing interval. This presentation will focus on the changes observed in the task distribution and control strategies among three Terminal Radar Approach Control (TRACON) controller participants and the workload associated with those changes across three conditions: Baseline, TSS, and DSAS. Results showed that the application of the DSTs in the TSS condition and those in the DSAS condition enabled a progressive reduction in the number of clearances issued and for the clearances to be issued earlier, further away from the airport. In doing so, there was also a greater use of speed control versus vectoring and altitude for schedule conformance in the TSS and DSAS conditions respectively and relative to Baseline. Workload also shifted in conjunction with the clearance distribution with an overall reduction across conditions. The changes in task and workload distribution enabled a shift from tactical to strategic control, which allowed for a more predictable and efficient delivery of arrivals.
Ensuring power grid resiliency, forecasting climate conditions, and optimization of transportation infrastructure are some of the many application areas where data is collected in both space and time. Spatiotemporal modeling is about modeling those patterns for forecasting future trends and carrying out critical decision-making by leveraging machine learning/deep learning. Once trained offline, field deployment of trained models for near real-time inference could be challenging because performance can vary significantly depending on the environment, available compute resources and tolerance to ambiguity in results. Users deploying spatiotemporal models for solving complex problems can benefit from analytical studies considering a plethora of system adaptations to understand the associated performance-quality trade-offs. To facilitate the co-design of next-generation hardware architectures for field deployment of trained models, it is critical to characterize the workloads of these deep learning (DL) applications during inference and assess their computational patterns at different levels of the execution stack. In this paper, we develop several variants of deep learning applications that use spatiotemporal data from dynamical systems. We study the associated computational patterns for inference workloads at different levels, considering relevant models (Long short-term Memory, Convolutional Neural Network and Spatio-Temporal Graph Convolution Network), DL frameworks (Tensorflow and PyTorch), precision (FP16, FP32, AMP, INT16 and INT8), inference runtime (ONNX and AI Template), post-training quantization (TensorRT) and platforms (Nvidia DGX A100 and Sambanova SN10 RDU). Overall, our findings indicate that although there is potential in mixed-precision models and post-training quantization for spatiotemporal modeling, extracting efficiency from contemporary GPU systems might be challenging. Instead, co-designing custom accelerators by leveraging optimized High Level Synthesis frameworks (such as SODA High-Level Synthesizer for customized FPGA/ASIC targets) can make workload-specific adjustments to enhance the efficiency.
AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.
Scientific workflows increasingly involve both HPC and machine-learning tasks, combining MPI-based simulations, training, and inference in a single execution. Launchers such as Slurm’s srun constrain concurrency and throughput, making them unsuitable for dynamic and heterogeneous workloads. We present a performance study of RADICAL-Pilot (RP) integrated with Flux and Dragon, two complementary runtime systems that enable hierarchical resource management and high-throughput function execution. Using synthetic and production-scale workloads on Frontier, we characterize the task execution properties of RP across runtime configurations. RP+Flux sustains up to 930 tasks/s, and RP+Flux+Dragon exceeds 1,500 tasks/s with over 99.6% utilization. In contrast, srun peaks at 152 tasks/s and degrades with scale, with utilization below 50%. For IMPECCABLE.v2 drug discovery campaign, RP+Flux reduces makespan by 30–60% relative to srun/Slurm and increases throughput more than four times on up to 1,024. These results demonstrate hybrid runtime integration in RP as a scalable approach for hybrid AI-HPC workloads.
The excess thermal energy produced by nuclear power plants (NPPs) during low electricity demands can be utilized in industrial processes, such as hydrogen production, through a thermal power dispatch (TPD) system. Initial testing of the first iteration of a single-train TPD design with a manual control mode at the Idaho National Lab (INL) revealed a high operator workload and degraded control capability. The current study evaluated the impact of an enhanced dual-train TPD design on operators’ subjective mental workload and evolution task-time while completing two operating scenarios in manual and automatic control modes. The results showed no statistically significant difference between participants’ mental workload using both control modes. Evolution time in automatic control mode took a shorter time than in manual control, with participants completing all evolutions in less than the 10-min set as the design specification limit. The shorter evolution time is discussed within the context of plant safety and operational efficiency.
The LANL Surveillance Workload Divinator (SWORD) is a high-level decision support tool that reproduces and predicts the annual values for stockpile and production disassembly and inspections (D&I) and rebuilds. SWORD, a spreadsheet-based tool, can be used to assess the surveillance workload under various planning scenarios. Results are shown graphically and updated dynamically for each weapon system or for the entire stockpile. Surveillance workload is calculated from stockpile surveillance guidance1 and from Production and Planning Directive (P&PD) data.
The National Energy Research Scientific Computing Center (NERSC), as the high-performance computing (HPC) facility for the Department of Energy’s Office of Science, recognizes the essential role of quantum computing in its future mission. In this report, we analyze the NERSC workload and identify materials science, quantum chemistry, and high-energy physics as the science domains and application areas that stand to benefit most from quantum computers. These domains jointly make up over 50% of the current NERSC production workload, which is illustrative of the impact quantum computing could have on NERSC’s mission going forward. We perform an extensive literature review and determine the quantum resources required to solve classically intractable problems within these science domains. This review also shows that the quantum resources required have consistently decreased over time due to algorithmic improvements and a deeper understanding of the problems. At the same time, public technology roadmaps from a collection of ten quantum computing companies predict a dramatic increase in capabilities over the next five to ten years. Our analysis reveals a significant overlap emerging in this time frame between the technological capabilities and the algorithmic requirements in these three scientific domains. We anticipate that the execution time of large-scale quantum workflows will become a major performance parameter and propose a simple metric, the Sustained Quantum System Performance (SQSP), to compare system-level performance and throughput for a heterogeneous workload.
This was an experimental study to develop an improved methodology for measuring workload in an information processing task and to assess the effects of shift length and communication density (rate of information flow) on the ability to process and classify verbal messages. Each of twelve subjects was exposed to combinations of three shift lengths and two communication densities in a counterbalanced, repeated measurements experimental design. Results indicated no systematic variation in task performance measures or in other dependent measures as a function of shift length or communication density. This is attributed to the absence of a secondary loading task, an insufficiently taxing work schedule, and the lack of psychological stress. Subjective magnitude estimates of workload showed fatigue (and to a lesser degree, tension) to be a power function of shift length. Estimates of task difficulty and fatigue were initially lower but increased more sharply over time under low density than under high density conditions. An interpretation of findings and recommedations for furture research are included. This research has major implications to human workload problems in information processing of air traffic control verbal data.
The techniques required to produce and validate six detailed task timeline scenarios for crew workload studies are described. Specific emphasis is given to: general aviation single pilot instrument flight rules operations in a high density traffic area; fixed path metering and spacing operations; and comparative workload operation between the forward and aft-flight decks of the NASA terminal control vehicle. The validation efforts also provide a cursory examination of the resultant demand workload based on the operating procedures depicted in the detailed task scenarios.
The data used to validate a seven time line analysis of forward flight deck autopilot mode for the pilot and copilot for NASA B737 terminal configured vehicle are presented. Demand workloads are given in two forms: workload histograms and workload summaries (bar graphs). A report showing task length and task interaction is also presented.
This paper briefly reviews the problems of defining and measuring the 'mental workload' of aircraft pilots and other human operators of complex dynamic systems. Of the alternative approaches the author indicates a clear preference for the use of subjective scaling. Some recent experiments from MIT and elsewhere are described which utilize subjective mental workload scales in conjunction with human decision and control tasks in the laboratory. Finally a new three-dimensional mental workload rating scale, under current development for use by IFR aircraft pilots, is presented.
A multidimensional bipolar-adjective rating scale is employed as a subjective measure of operator workload in the performance of a one-axis tracking task. The rating scale addressed several dimensions of workload, including cognitive, physical, and perceptual task loading as well as fatigue and stress effects. Eight subjects performed a one-axis tracking task (with six levels of difficulty) and rated these tasks on several workload dimensions. Performance measures were tracking error RMS (root-mean square) and the standard deviation of control stick output. Significant relationships were observed between these performance measures and skill required, task complexity, attention level, task difficulty, task demands, and stress level.
A literature review was conducted to assess the basis of and techniques for physiological assessment of mental workload. The study findings reviewed had shortcomings involving one or more of the following basic problems: (1) physiologic arousal can be easily driven by nonworkload factors, confounding any proposed metric; (2) the profound absence of underlying physiologic models has promulgated a multiplicity of seemingly arbitrary signal processing techniques; (3) the unspecified multidimensional nature of physiological "state" has given rise to a broad spectrum of competing noncommensurate metrics; and (4) the lack of an adequate definition of workload compels physiologic correlations to suffer either from the vagueness of implicit workload measures or from the variance of explicit subjective assessments. Using specific studies as examples, two basic signal processing/data reduction techniques in current use, time and ensemble averaging are discussed.
An operator workload measurement methodology is presented which will be used in support of the Mark IVA operational test and evaluation plan. Three operator workload measures are suggested: operator ratings, primary task work measures, and information processing time measures. A method of validating the workload measures using secondary task work measures is presented. It is expected that operations testing using these measures will assist in establishing the time required to perform essential operational activities and will indicate high risk operations areas due to potential operator overload.
The objective was to determine whether the Sternberg item-recognition task, employed as a secondary task measure of spare mental capacity for flight handling qualities (FHQ) simulation research, could help to differentiate between different flight-control conditions. FHQ evaluations were conducted on the Vertical Motion Simulator at Ames Research Center to investigate different primary flight-control configurations, and selected stability and control augmentation levels for helicopters engaged in low-level flight regimes. The Sternberg task was superimposed upon the primary flight-control task in a balanced experimental design. The results of parametric statistical analysis of Sternberg secondary task data failed to support the continued use of this task as a measure of pilot workload. In addition to the secondary task, subjects provided Cooper-Harper pilot ratings (CHPR) and responded to workload questionnaire. The CHPR data also failed to provide reliable statistical discrimination between FHQ treatment conditions; some insight into the behavior of the secondary task was gained from the workload questionnaire data.