SEARCH · Engineering Papers

Results for “Digitizer Evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2024

A Full Accounting of the Visible Mass in SDSS MaNGA Disk Galaxies

We present a study of the ratio of visible mass to total mass in spiral galaxies to better understand the relative amount of dark matter present in galaxies of different masses and evolutionary stages. Using the velocities of the Hα emission line measured in spectroscopic observations from the Sloan Digital Sky Survey (SDSS) MaNGA Data Release 17 (DR 17), we evaluate the rotational velocity of over 5500 disk galaxies at their 90% elliptical Petrosian radii, R 90 . We compare this to the velocity expected from the total visible mass, which we compute from the stellar, H i, H 2 , and heavy metals and dust masses. H 2 mass measurements are available for only a small subset of galaxies observed in SDSS MaNGA DR17, so we derive a parameterization of the H 2 mass as a function of absolute magnitude in the r band using galaxies observed as part of SDSS DR7. With these parameterizations, we calculate the fraction of visible mass within R 90 that corresponds to the observed velocity. Based on statistically analyzing the likelihood of this fraction, we conclude that the null hypothesis (no dark matter) cannot be excluded at a confidence level better than 95% within the visible extent of the disk galaxies. We also find that when all mass components are included, the ratio of visible to total mass within the visible extent of star-forming disk galaxies increases with galaxy luminosity.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2026

SoK: What does it Mean to Benchmark Database Forensics?

Relational Database Management Systems are the backbone of modern enterprises and public-sector services, and are thus frequent targets of security incidents, insider threats, and thorough regulatory audits. Consequently, databases have become key sources of digital evidence, requiring investigators to reconstruct past activity from audit logs, transaction logs, and backups. Although benchmarking frameworks such as those developed by the Transaction Processing Performance Council (TPC) are widely used to evaluate database performance, they do not capture forensic requirements such as evidentiary completeness, tamper-evidence, chain of custody, or regulatory compliance under GDPR and CCPA. This survey examines the emerging domain of forensic database benchmarking. We gathered prior research on database forensics, secure logging, and tamper-evident data structures; we analyze modern forensic-ready features in commercial and open-source systems (SQL Server Ledger, Oracle Blockchain Tables, PostgreSQL pgAudit, Db2 Audit, Aurora Database Activity Streams, Oracle Real Application Security and IBM Guardium) and assess why existing benchmarks are insufficient. We propose forensic workloads, metrics, and methodologies that incorporate adversarial stressors, deleted-record recovery, and backup analysis. We also identify open research problems and call for a community-driven forensic benchmark suite. The result is an idea for evaluating not only database performance but also forensic soundness, bridging the gap between system engineering, compliance, and digital investigations.

Lenard, Ben↗

DOE OSTI2024

Identification of Distorted Gamma-Ray Signature Patterns Using Digital Filtering and Auto-Associative Memory Implemented with a Hopfield Neural Network

The detection and identification of radioactive sources in search applications involve analyzing passive gamma-ray emissions from high-level radioactive materials. This process uses a mobile detector-spectrometer in a complex field test environment. Recently, the use of artificial intelligence for gamma-ray spectrum analysis has shown promising results. However, challenges persist in identifying isotopic signatures from spectral measurements that may be distorted due to source shielding, random variations in natural radioactive background, or insufficient measurement time to obtain clear spectral lines. Here, this paper presents a novel intelligent signature recognition method that combines digital filtering techniques with an artificial Hopfield Neural Network (HNN). The HNN leverages auto-associative memory to store training sample patterns and match them with incoming gamma spectra from distorted sources. It restores the testing sources’ measurements by finding the closest matching signature patterns in the spectral library. Before HNN recognition, the measured spectrum undergoes preprocessing with a digital image filter to reduce fluctuations. Performance of the proposed method is evaluated using a set of gamma-ray spectra measured with a sodium iodide detector. The data collected include measurements from six pure samples: 241 Am, 60 Co, 137 Cs, 192 Ir, 239 Pu, and 235 U, which are used for training and validation (i.e. six cases). Additionally, the data set contains 24 distorted synthesized sources with various fluctuating backgrounds. Test results demonstrate the potential of the proposed method to accurately recognize the correct isotope with high precision, achieving an accuracy rate exceeding 85%. Furthermore, the proposed method exhibits superior performance compared to the conventional multiple regression fitting and simple feedforward neural network methods.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

42 ENGINEERING↗

DOE OSTI2024

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

actuated signal control↗

DOE OSTI2024

Temporal and Spatial Evolution of Non-Elastic Strain Accumulation in Stanstead Granite During Brittle Creep

Understanding the long-term behavior of brittle rocks requires fundamental consideration of time-dependent strain evolution and brittle creep processes. Previous studies have evaluated sub-critical crack growth during time-dependent deformation and damage evolution in brittle rocks; however, there is an incomplete knowledge of how damage evolves spatially and temporally within the body of intact rocks, where distributed regions of damage interact and coalesce during creep. This paper presents laboratory research focusing on evaluating brittle creep damage processes in Stanstead granite (SG) using 2-dimensional digital image correlation (2D-DIC). In the laboratory, the prismatic SG specimens were loaded beyond an estimated Crack Damage stress threshold (CD) level and then maintained a constant stress to initiate the creep process. DIC was used to characterize full-field spatiotemporal strain evolution, which was then interpreted in the context of local regions of “damage”, determined according to a strain-based criterion. Here, a method was proposed for identifying “existing” and “new” damage regions over specified intervals during the test, followed by spatial clustering of these regions to assess their spatiotemporal evolution. The clustering analysis results demonstrated the extension of existing damage regions was the main damage process during brittle creep, which is consistent with existing models of sub-critical crack growth. In addition, temporal analysis of tensile and shear strains on a point-by-point basis revealed both new damage formation and the strain concentration within existing damaged regions significantly contribute to overall specimen strain during primary creep. In contrast, during secondary creep, increases in specimen deformation are influenced by the accumulation of strains within already damaged regions.

58 GEOSCIENCES↗

DOE OSTI2026

Are the U.S. Biorefineries Over the Hurdle of 2000 Ton Daily Throughput Yet?

The efficient utilization of lignocellulosic biomass for biofuel and biochemical production is hindered by material handling issues such as clogging and segregation among other challenges. Preprocessing methods such as drying, screening, and milling have improved conversion yield but have not sufficiently enhanced flowability, especially herbaceous biomass. The poor flowability of herbaceous biomass is rooted in some particle attributes that remain less altered by those methods, e.g., irregular particle shape, high roughness, and high compressibility, making it hard to scale up throughput to a key benchmark for a biorefinery – 2000 ton per day. Applying additional preprocessing methods like pelletization and torrefaction to drastically change those particle attributes can improve flow and handling but has not been comprehensively verified through test. The flowability of herbaceous biomass feedstock formats generated by three different preprocessing methods was recently assessed at Idaho National Laboratory’s Biomass Feedstock National User Facility: first, loose particles size reduced from as-received materials; second, pellets produced from an efficient densification process; and third, powders milled from torrefied pellets. Benchmarking tests including static angle of repose, basic flow energy measured in a powder rheometer, and discharge flow in an adjustable hopper, were conducted to evaluate those feedstock formats. Beyond the capacity of existing experimental apparatuses, a digital engineering approach involving flow simulations and AI models were used to identity the material attributes and processing parameters that have dominant influences on flow throughput. Techno-economic analysis focusing on hopper flow as a typical material handling operation was conducted for those feedstock formats. Perspectives will be discussed on whether the 2000-ton daily throughput for a biorefinery is achievable at an acceptable cost by using any of the tested preprocessing methods.

09 - BIOMASS FUELS↗

DOE OSTI2026

A simulation framework for evaluating electronic order workflows in integrated health records

Electronic health record (EHR) systems are critical to modern healthcare delivery, yet the dynamic workflows that govern electronic order processing remain underexplored. Inefficiencies in these digital pathways can cause delays in care, repetitive workloads, and even patient harm. This study presents a discrete-event simulation framework used to reconstruct and evaluate EHR-based order workflows in a large integrated healthcare system. Using real-world data extracted from the Veterans Health Administration’s Corporate Data Warehouse, the authors mapped order events to standardized state transitions and modeled their progression across different facilities of varying complexity levels. After being calibrated with empirical distributions of transition times and validated against observed time-in-system metrics, the simulation demonstrates close alignment with historical performance. Scenario analyses reveal that resource capacity constraints significantly amplify the impact of electronic order surges, which are reflected in the disproportionate growth in backlogs and processing delays. Adjustments in transition probabilities further increased recirculation and extended workflow paths. Network-based analysis identified Reserved, InProgress, and Completed as structurally critical states that function as hubs within the process network but the transitions in-between also act as major bottlenecks. These results showcased the effectiveness of simulation-based approaches in monitoring EHR order processing performance and evaluating consequences of workflow changes on healthcare network resources planning. The proposed simulation framework provides a scalable data-driven tool to support operational decision-making and improve the efficiency of electronic order management in complex healthcare environments.

Engineering↗

DOE OSTI2026

Screening and qualification methodology for SiC end plug processing methods

Deployment of SiC-ceramic-based fuel cladding for light water reactors requires a hermetic end plug–to–cladding joint that can withstand neutron irradiation during normal operation and maintain integrity during design-basis accidents. Reactor experiments have shown that some SiC composite tubes with SiC end plugs can retain hermeticity after irradiation. However, achieving consistent joint performance under irradiation remains a key challenge. Resolving this issue is essential to enable integral irradiation testing and to demonstrate fuel integrity under commercial-reactor irradiation conditions. This report aims to: (1) provide guidance for designing radiation-tolerant end plug joints for SiC cladding; (2) demonstrate experimental methods to detect processing defects that are unstable under neutron irradiation at light-water-reactor-relevant temperatures and doses; and (3) outline a step-by-step approach for designing and conducting reactor experiments to screen joining methods. The resulting data will be used to improve joint processing and to define critical defect types and sizes that must be detected and eliminated through non-destructive evaluation for quality assurance. Based on prior irradiation experiments at the High Flux Isotope Reactor, differential swelling among the cladding, bonding layer, and end plug was identified as an underlying mechanism for irradiation-induced joint degradation. Accordingly, this effect must be considered in the design of radiation-tolerant joining techniques. In this work, miniature SiC end plug joint specimens irradiated during the previous project were analyzed using X-ray computed tomography to characterize the joint microstructure. Digital volume correlation of the tomography data quantified radiation-induced microstructural changes and enabled evaluation of defect-related risks. Finally, ongoing neutron irradiation efforts using larger specimen volumes are presented. These efforts aim to statistically assess joint performance and to build a microstructure–performance (e.g., leak-tightness) dataset to inform processing improvements and quality control.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Enhancing the Operational Resilience of Advanced Reactors with Digital Twins by Recurrent Neural Networks

Because of a lack of operation data during abnormal and accident scenarios, along with the existence of uncertainty in the evaluation model for transient and accident analysis, the established abnormal and emergency operating procedures can be biased in characterizing the reactor states and ensuring operational resilience. To improve state awareness and ensure operational flexibility for minimizing effects on the system due to anomaly, digital twin (DT) technology is suggested to support operator's decision-making by effectively extracting and using knowledge of the current and future plant states from the knowledge base. To demonstrate DT's capability for recovering the complete states of reactors and for predicting the future reactor behaviors, this paper develops and assesses both the diagnosis and prognosis DTs in a nearly autonomous management and control system for an Experimental Breeder Reactor-II simulator during different loss-of-flow scenarios.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

Human and Technology Integration Evaluation of Advanced Automation and Data Visualization

While the existing United States (U.S.) light water reactors are highly reliable, safe, and provide a significant proportion of carbon-free electricity, the cost of operating and maintaining them has become less competitive compared to other electricity generating sources. The reason for the gap in operating and maintenance (O&M) costs can be at least in part attributed to the advent of new digital technologies that other electricity generating industries are currently using. Advanced capabilities including digital instrumentation and control (I&C) systems, advanced automation and analytics, and greater span of data integration (i.e., connectedness) across these non-nuclear plants has transformed the way work is performed and ultimately given them a competitive advantage in terms of the cost required for operating, maintaining, and supporting them. To reduce O&M cost and address obsolescence of the aging I&C infrastructure of the existing U.S. light water reactors, the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program Plant Modernization Pathway is conducting targeting multidisciplinary research that 1) delivers a sustainable business model to enable a cost-competitive U.S. nuclear industry and 2) is developing technology modernization solutions that address aging and obsolescence challenges. The work described in this report supports these two objectives and describes the demonstration of human and technology integration across recent industry collaborations to support their large-scale digital I&C modifications. This technical report describes the demonstration of the human and technology integration methodology in performing full-scale performance-based human-in-the-loop tests to evaluate plant-specific advanced automation and data visualization applications within these collaborators’ digital modifications. This technical report also documents future applications of human and technology integration that expand beyond main control room modernization and digital I&C upgrades, which have been a central focus to date. Thus, this technical report discusses how to implement human and technology integration across new business opportunities and how to develop an evaluation plan that defines measures and criteria, and documents key assumptions to support full plant modernization.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2022

Atikokan Digital Twin: Machine learning in a biomass energy system

The Atikokan Generating Station, operated by Ontario Power Generation, has a 200 MW, biomass-fired tower boiler that operates on a dispatch schedule with a five-minute cycle. The boiler is generally operated in the range of 40–100 MW using two of five burner levels. In order to optimize boiler performance, we propose the implementation of a unique digital twin. Our digital twin abstraction couples Bayesian inference from science-based models and from observations (machine learning) with decision theory to predict operating-variable set points that optimize the physical asset (the boiler) in the presence of uncertainty (artificial intelligence). We focus this paper on the continuous Bayesian machine learning part of the Atikokan Digital Twin; we discuss decision theory in a companion paper. We identify and learn about 12 operational, model, and measured-output parameters and their uncertainties from high-fidelity, science-based simulations of the Atikokan boiler and from the observed measurements at the power plant. Since the goal of the Atikokan Digital Twin is to implement it online in real time, we require fast function evaluations for the quantities of interest extracted from the simulations in the Bayesian analysis. We use Gaussian process regression/interpolation to create accurate, robust surrogate models. We define the Bayesian priors and likelihood function and solve for the posterior distributions of the 12 parameters. Here we then propagate these distributions (i.e., parameters with uncertainty) into the predicted distributions of 790 quantities of interest to learn about the relative importance of various sources of error including experimental, model, and operating-parameter errors.

09 BIOMASS FUELS↗

DOE OSTI2020

On the Formalization of Development and Assessment Process for Digital Twins in the Nearly Autonomous Management and Control System

In recent years, the autonomous control system has been encouraged in advanced reactors for restoring economic viability, simplifying the operation and maintenance, and enabling remote-site power generations [1]. Since the reactor is expected to be operated for a long period of time with a limited number of individuals onsite, it is recommended that the autonomous control system should have access to very realistic models of the state of processes in the whole lifecycle, together with these process behaviors in interaction with their environment in the real world. As a result, digital twin (DT) technology is suggested in autonomous control systems. DT is defined as a digital representation of a physical object or system, which contains a record for the histories of loads, operation and maintenance status, predictions for the near-term transient of important state variables, and decision-making process [2]. Since machine learning (ML) can recognize patterns within a complex system in real-time applications, it has been used to build DTs in the autonomous control systems for advanced reactors. Meanwhile, due to the rareness of operation data in accident scenarios, the development and assessment of DTs is expected to be mainly driven by simulations. Although the capability and feasibility of ML-based DTs are recognized in improving the safety and efficiency of reactor control, a major concern from the regulatory commission and the nuclear industry is whether the information from a DT is developed and assessed in accordance with expectation and requirements by the target decision. Such concerns not only affect the acceptance criteria for DTs, but also values that can be extracted from DTs and autonomous control system during operations. Inspired by the success of formal methods in improving the reliability and robustness of computer programming and software development, it is suggested that the development and assessment process (DAP) for both separate DTs and integral control system should be formalized in a transparent, consistent, and improvable manner. In this study, a digital-twin development and assessment process (DT-DAP) is proposed by adapting the evaluation model development and assessment process (EMDAP) [3] to requirements by the autonomous control system, ML algorithms, and DT technology. To demonstrate the framework, a baseline nearly autonomous management and control (NAMAC) system with ML-based DTs for diagnosis and prognosis is developed and assessed based on the framework. It is found that with selected testing methods and techniques, the DT-DAP can help identify errors in DTs and NAMAC which would otherwise be left unverified. Meanwhile, it is found that the DT-DAP can improve the DTs and NAMAC by continuously learning and iterating through different elements.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Evaluation of pebble scanning strategies for fuel qualification by simple simulated radiography

Reactor designs using tristructural-isotropic (TRISO) particles in pebble fuel forms have been developed in multiple programs and are now being designed and deployed by several companies around the world. Fuel pebbles for these designs commonly contain an inner fueled zone packed with TRISO particles and a fuel-free zone on the surface of the pebble to protect the particles within. Therefore, qualification of pebbles for use in a reactor commonly requires determination that there are no TRISO particles within a given minimum distance to the pebble surface. This determination may be made non-destructively by pebble radiography, in which a series of x-ray projections of the pebble are acquired. In this work, a method to create and image representative digital pebbles was developed to allow for rapid testing of various pebble radiography strategies to evaluate their effectiveness in detecting escapee particles. Using this method, imaging strategies in two and three dimensions were tested with fixed and adaptive angular distributions. Here it was found that an adaptive two-dimensional pebble scanning strategy was the most effective for fuel pebble qualification.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2025

High-power test of a C-band linear accelerating structure with an RFSoC-based LLRF system

Normal conducting linear particle accelerators consist of multiple rf stations with accelerating structure cavities. Low-level rf (LLRF) systems are employed to set the phase and amplitude of the field in the accelerating structure and to compensate for the pulse-to-pulse fluctuation of the rf field in the accelerating structures with a feedback loop. The LLRF systems are typically implemented with analog rf mixers, heterodyne-based architectures, and discrete data converters. There are multiple rf signals from each of the rf stations, so the number of rf channels required increases rapidly with multiple rf stations. With a large number of rf channels, the footprint, component cost, and system complexity of the LLRF hardware will increase significantly. To meet the design goals of being compact and affordable for future accelerators, we have designed the next-generation LLRF (NG-LLRF) with a higher integration level based on RFSoC technology. The NG-LLRF system samples rf signals directly and performs rf mixing digitally. Further, the NG-LLRF has been characterized in loopback mode to evaluate the performance of the system and has also been tested with a standing-wave accelerating structure, a prototype for the Cool Copper Collider (C 3 ) with a peak rf power level up to 16.45 MW. The loopback test demonstrated amplitude fluctuation below 0.15% and phase fluctuation below 0.15°, which are considerably better than the requirements of C 3 . The rf signals from the different stages of the accelerating structure at different power levels are measured by the NG-LLRF, which will be critical references for the control algorithm designs. The NG-LLRF also offers flexibility in waveform modulation, so we have used rf pulses with various modulation schemes, which could be useful for controlling some of the rf stations in accelerators. In this paper, the high-power test results at different stages of the test setup will be summarized, analyzed, and discussed.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Reference Site Conditions for Floating Wind Arrays in the United States

17 WIND ENERGY↗

DOE OSTI2024

Reference Site Condition Datasets for Floating Wind Arrays in the United States

Floating offshore wind farm design is highly site-specific, requiring detailed information about the specific conditions of a project area for realistic design studies. Unfortunately, publicly available site condition data for potential floating offshore wind project sites in the United States is scarce. To support U.S. offshore wind research, we developed reference site condition datasets, including metocean and seabed information, for four potential floating wind project areas in the U.S.: Humboldt Bay, Morro Bay, the Gulf of Maine, and the Gulf of Mexico. These datasets were compiled using publicly available data. Our metocean analysis, covering wind, waves, and surface currents, utilized measurement data from 2000 to 2020. Sources included the National Renewable Energy Laboratory’s National Offshore Wind Dataset for wind data, National Data Buoy Center buoys for wave data, and the High Frequency Radar Network for surface currents. These data were integrated into hourly time series used to compute extreme return periods up to 500 years, monthly statistics, and joint probability clusters for fatigue analysis. Soil conditions were evaluated using the usSEABED database and bathymetry grids were interpolated from the NCEI Digital Elevation Model Global Mosaic. Further information on the datasets and how they were created can be found in: Biglu, M., M. Hall, E. Lozon, S. Housner. 2024. Reference Site Conditions for Floating Wind Arrays in the United States. Golden, CO: National Renewable Energy Laboratory (NREL). NREL/TP-5000-89897. The data are also available at: https://github.com/FloatingArrayDesign/SiteConditions The content of each dataset is as follows: _NOW23_wind.txt: Hourly NOW-23 wind data up to a height of 400 meter. _metocean_1hr.txt: Hourly time series including wind, wave, surface current and temperature data. _Summary.xlsx: Metocean data, including extreme values, joint probability distributions and monthly statistics. _usSEABED_soil.csv: Extract of the usSEABED database for this specific site. _bathymetry_200m.txt (and 500m, 1000m): Gridded seabed depth data.

16 TIDAL AND WAVE POWER↗

DOE OSTI2020

Data Collection and Analysis Challenges and Mitigation Strategies for Quantitative Human Factors Research Studies in Nuclear Power Plant Modernization

The United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program Plant Modernization Pathway is conducting targeted research and development (R&D) to address aging and reliability concerns with the legacy instrumentation and control and related information systems of the U.S. LWR fleet. In this effort, the application of human factors engineering (HFE) provides an important role in ensuring new digital plant technologies enable broad innovation and business improvement with continued operational safety. Evaluation is a key activity in HFE, which often occurs iteratively through the system design lifecycle. While qualitative methods are important in collecting information of how users perform tasks through observations, quantitative methods are equally important in assessing system design based on performance. In collecting and analyzing this quantitative data, there are notable challenges in the nuclear HFE domain that may threaten the validity and reliability of the inferences made in these studies. Notable challenges include small sample size and limited resources, large error variance and small effect size, an “adding test features to losing degrees of freedom” dilemma, non-normal distribution, and heterogeneity of variance. The results in control room usability studies are often statistically non-significant, which makes it hard to interpret. This work discusses these challenges across different scientific viewpoints and provides real-world examples of these challenges in practice. Collectively, the objective of this work is to position these challenges to the larger data science community as a means of identifying future opportunities to address these issues.

99 GENERAL AND MISCELLANEOUS↗