SEARCH · Engineering Papers

Results for “gap analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

All sources NASA NTRS DOE OSTI

NASA NTRS2011

Thermoelectric Performance Enhancement by Surrounding Crystalline Semiconductors with Metallic Nanoparticles

Direct conversion of thermal energy to electricity by thermoelectric (TE) devices may play a key role in future energy production and utilization. However, relatively poor performance of current TE materials has slowed development of new energy conversion applications. Recent reports have shown that the dimensionless Figure of Merit, ZT, for TE devices can be increased beyond the state-of-the-art level by nanoscale structuring of materials to reduce their thermal conductivity. New morphologically designed TE materials have been fabricated at the NASA Langley Research Center, and their characterization is underway. These newly designed materials are based on semiconductor crystal grains whose surfaces are surrounded by metallic nanoparticles. The nanoscale particles are used to tailor the thermal and electrical conduction properties for TE applications by altering the phonon and electron transport pathways. A sample of bismuth telluride decorated with metallic nanoparticles showed less thermal conductivity and twice the electrical conductivity at room temperature as compared to pure Bi2Te3. Apparently, electrons cross easily between semiconductor crystal grains via the intervening metallic nanoparticle bridges, but phonons are scattered at the interfacing gaps. Hence, if the interfacing gap is larger than the mean free path of the phonon, thermal energy transmission from one grain to others is reduced. Here we describe the design and analysis of these new materials that offer substantial improvements in thermoelectric performance.

Kim, Hyun-Jung↗

NASA NTRS2003

Advanced Rainbow Solar Photovoltaic Arrays

Photovoltaic arrays of the rainbow type, equipped with light-concentrator and spectral-beam-splitter optics, have been investigated in a continuing effort to develop lightweight, high-efficiency solar electric power sources. This investigation has contributed to a revival of the concept of the rainbow photovoltaic array, which originated in the 1950s but proved unrealistic at that time because the selection of solar photovoltaic cells was too limited. Advances in the art of photovoltaic cells since that time have rendered the concept more realistic, thereby prompting the present development effort. A rainbow photovoltaic array comprises side-by-side strings of series-connected photovoltaic cells. The cells in each string have the same bandgap, which differs from the bandgaps of the other strings. Hence, each string operates most efficiently in a unique wavelength band determined by its bandgap. To obtain maximum energy-conversion efficiency and to minimize the size and weight of the array for a given sunlight input aperture, the sunlight incident on the aperture is concentrated, then spectrally dispersed onto the photovoltaic array plane, whereon each string of cells is positioned to intercept the light in its wavelength band of most efficient operation. The number of cells in each string is chosen so that the output potentials of all the strings are the same; this makes it possible to connect the strings together in parallel to maximize the output current of the array. According to the original rainbow photovoltaic concept, the concentrated sunlight was to be split into multiple beams by use of an array of dichroic filters designed so that each beam would contain light in one of the desired wavelength bands. The concept has since been modified to provide for dispersion of the spectrum by use of adjacent prisms. A proposal for an advanced version calls for a unitary concentrator/ spectral-beam-splitter optic in the form of a parabolic curved Fresnel-like prism array with panels of photovoltaic cells on two sides (see figure). The surface supporting the solar cells can be adjusted in length or angle to accommodate the incident spectral pattern. An unoptimized prototype assembly containing ten adjacent prisms and three photovoltaic cells with different bandgaps (InGaP2, GaAs, and InGaAs) was constructed to demonstrate feasibility. The actual array will consist of a lightweight thin-film silicon layer of prisms curved into a parabolic shape. In an initial test under illumination of 1 sun at zero airmass, the energy-conversion efficiency of the assembly was found to be 20 percent. Further analysis of the data from this test led to a projected energy conversion efficiency as high as 41 percent for an array of 6 cells or strings (GaP, AlGaAs, InGaP2, GaAs, and two different InGaAs cells or strings).

Mardesich, Nick↗

NASA NTRSDate not supplied

Developing A Continuous Ozone Record Through the SAGE and Aura Missions With NASA Reanalysis Products

During the last quarter of the 20th century, the Stratospheric Aerosol and Gas Experiment (SAGE) missions were crucial in monitoring the loss and the subsequent recovery of the stratospheric ozone layer. Due to the employed solar occultation and self-calibration method, the SAGE monitors have produced stable data throughout the lifetime of each instrument. However, over ten years passed between the end of the SAGE II and SAGE III/M3M missions in 2005 and the launch of SAGE III/ISS instrument in 2017, leaving a gap in the data that much be bridged in order to assess the trends in the ozone record. Reanalysis products, such as the Modern-Era Retrospective analysis for Research and Applications, version 2 (MERRA-2), are attractive candidates for trend analysis due to the statistically optimized combination of multiple observing systems and the regular temporal and spatial coverage. In this study, we explore using the SAGE records to develop a stable reanalysis data product, suitable for trend analysis, from the start of the SAGE II record in 1984 through the present. Changes in the assimilated observation systems can introduce discontinuities within the MERRA-2 ozone record, such as in 2004 when the MERRA-2 system shifted from assimilating ozone retrievals collected by SBUV instruments to those collected by instruments onboard the Aura satellite. We follow the radiative transfer procedure outlined by Wargan et al. (2018) to address discontinuities in the MERRA-2 ozone dataset at the 2004 transition and during the Aura record. SAGE II ozone profiles are used to address discontinuities in upper stratospheric ozone associated with changes in the MERRA-2 meteorological observing system in 1998 and 1995. Lastly, we will use the resulting bias-corrected MERRA-2 ozone fields to assess the relative performance of the data from different SAGE sensors.

SAGE↗

NASA NTRSDate not supplied

Using Artificial Intelligence and Machine Learning to Enhance Mission Design and Operations of the Habitable Worlds Observatory (HWO)

One key aspect in the development of HWO is the early deployment of artificial intelligence (AI) and machine learning (ML) to enhance mission science and operations. Our subtask group is part of the HWO AI/ML working group and focuses on AI and ML for mission operations. Our task group seeks to educate other HWO working groups about AI and ML capabilities for mission operations, investigate how to bridge technology gaps, and enable new capabilities particularly in the areas of observational scheduling, instrument health monitoring, and downlink operations. We focus on mission tasking / scheduling both for mission analysis in development and operations. AI and ML for mission scheduling includes: tools to support proposal calls and review, ensuring fairness in calls for proposals, community peer reviews and ease workloads, as well as in-flight and ground software development (e.g., using natural language processing (NLP) to support process automation from requirements). AI and ML for the mission’s development and operations include 1) anomaly detection and prediction (from onboard and ground based tools) to monitor the spacecraft’s health, 2) ground-based automated scheduling for mission operations including long-term and short-term planning and maintenance, and 3) flight system flexible execution (as flight proven for Spitzer and JWST) to enable robust execution despite execution variations, and 4) data analysis for prioritization (e.g., real-time data evaluation leading to autonomous actions and adjustments, high-priority identification, onboard data compression, etc.). Incorporation of ML and AI will enable HWO to address the major science questions related to exoplanet characterization, general astrophysics, and solar system exploration and also extend the boundaries of space mission technologies.

Mark Moussa↗

NASA NTRSDate not supplied

Real Time Nitrogen Monitoring and Quality Control System for NASA Astromaterials Collection

The Nitrogen system is a vital utility supporting the curation office within NASA's Astromaterials Re-search and Exploration Science (ARES) Division at the Johnson Space center (JSC). Since the Apollo samples were returned in 1969. It has played an essential role in preserving NASA’s astromaterial collections (now nine collections) by maintaining contamination-free, controlled environments required for long term curation and advanced scientific research. To meet the stringent purity requirements for contamination control, high purity modified Grade C Liquid Nitrogen (LN2) is delivered weekly by an industrial gas vendor to a 15,000-gallon tank located outside at NASA JSC near the ARES facility. This nitrogen is vital for creating an inert atmosphere to store the collection in, ensuring they are continuously shielded from contaminants and alterations through a constant N2purge. For most of the past 65 years deliveries of LN2 have required a CoA (Certificate of Analysis) prior to delivery. If those were not available, then LN2 deliver were sent to third-party laboratories for verification, creating an operational bottleneck and quality gap. To address this challenge and maintain rigorous quality control, NASA JSC Infrastructure and Astromaterials Acquisition & Curation Office initiated the development of an on-site Analytical Gas Sampling (AGS) Laboratory. Designed to streamline quality verification if a CoA unavailable, the AGS lab was constructed by a subcontractor in Newton, NJ. After a successful factory acceptance test conducted by the NASA ARES Infrastructure team, the lab was trans-ported to NASA JSC, where it was anchored and integrated into the site’s nitrogen pressure system using stainless steel, oxygen-cleaned tubing. The ARES nitrogen system was modified to provide parallel flows to the AGS lab for quality control and to the astromaterials collection labs for maintaining an inert environment. The AGS lab features two operational modes: (1)Primary: Gaseous Nitrogen (GN2) Quality Monitoring –The AGS system continuously samples the site’s GN2 supply every 15 minutes to ensure compliance with required purity levels. (2) Secondary: LN2 Quality Verification when a comprehensive CoA is not available, a cryogenic vaporizer converts LN2 to GN2, which is then analyzed for impurities: H₂, Ar, O₂, CO, CO₂, H₂O, and total hydrocarbons (THC). By significantly enhancing on-site analytical capabilities, the AGS lab eliminates reliance on external testing, ensures uninterrupted quality monitoring of the nitrogen pressure system, and reinforces NASA’s commitment to preserving pristine astromaterials under the highest standards of curation and research integrity.

glove box↗

DOE OSTI2022

A Technical-Economic Assessment of Brackish Water Treatment for Fossil Power Plant Cooling with Reduced Environmental Impact

The power sector accounts for approximately 40% of freshwater use in the U.S. Reusing non-traditional water resources is critical for alleviating water stress in the western United States. The purpose of this study is to conduct a techno-economic system analysis to evaluate the potential of brackish water treatment for wet cooling towers at existing thermo-electric generating units (EGUs). It aims to address current knowledge gaps in support of non-traditional water resources planning and the electric power industry's water decision- and policy making. An engineering-economic model was developed and combined with DuPont's water application value engine (WAVE) v1.82 to evaluate the viability of retrofitting reverse osmosis-based brackish water treatment systems to individual existing EGUs and then aggregating the unit-level results to the fleet level. In the base case, brackish water treatment can reduce freshwater consumption of electric generation units by 94–100% while increasing the cost of electric generation by approximately 10% and have marginal impacts on the units' parasitic loads. When zero liquid discharge systems are applied to brackish water brine treatment, the cost of electric generation and freshwater consumption savings and energy penalty to the power plant significantly increase. These costs are even higher than dry cooling deployment in lieu of wet cooling towers. The choice of freshwater savings between brackish water desalination and dry cooling deployment depends greatly on how the concentrated brines are managed. Multiple factors must be considered when supporting decision-making in water management and planning in power sectors, which include non-traditional water quantity and quality as well as trade-offs among technical performance, economics, and environmental impacts.

Wu, Zitao↗

NASA NTRSDate not supplied

Evaluating the Global Water and Energy Cycles in Reanalyses Using Ensemble Spread

Reanalyses provide continuous observation based global data over weather and climate time scales through observational assimilation. Ensemble forecasts are a component of modern data assimilation systems, providing information to the analysis process. Storing this spread over the duration of a climate reanalysis will provide insight to the variations of the model that underlies the reanalysis data. Global water and energy cycles (WECs) are a critical aspect of both weather and climate, yet there are some large gaps in the observation of key components as well as their representation in models and reanalyses. NASA’s Water and Energy cycle Studies (NEWS) program is developing a corrected and balanced budget data set that is closed at monthly and regional scales during the 21st century, so far. In this paper, we will evaluate reanalysis WEC components against the initial NEWS collection of observation, but integrate the analysis spread from the reanalysis data assimilation as a new perspective on the evaluation. The initial evaluation will cover a few more recent years using ERA5 10 member spread. As the GMAO’s production of the Modern-Era Retrospective analysis for Research and Applications 21st Century (MERRA-21C) progresses, the effort will include the MERRA-21C 32 member spread. Ultimately, we will explore the potential of the ensemble spread as a measure of uncertainty in the reanalysis.

Michael Bosilovich↗

NASA NTRS1986

Asymptotic analysis of corona discharge from thin electrodes

The steady discharge of a high-voltage corona is analyzed as a singular perturbation problem. The small parameter is the ratio of the length of the ionization region to the total gap length. By this method, current versus voltage characteristics can be calculated analytically.

Durbin, P. A.↗

NASA NTRS2013

Preparing for Humans at Mars, MPPG Updates to Strategic Knowledge Gaps and Collaboration with Science Missions

The Mars Program Planning Group (MPPG) was an agency wide effort, chartered in March 2012 by the NASA Associate Administrator for Science, in collaboration with NASA's Associate Administrator for Human Exploration and Operations, the Chief Scientist, and the Chief Technologist. NASA tasked the MPPG to develop foundations for a program-level architecture for robotic exploration of Mars that is consistent with the President's challenge of sending humans to the Mars system in the decade of the 2030s and responsive to the primary scientific goals of the 2011 NRC Decadal Survey for Planetary Science. The Mars Exploration Program Analysis Group (MEPAG) also sponsored a Precursor measurement Strategy Analysis Group (P-SAG) to revisit prior assessments of required precursor measurements for the human exploration of Mars. This paper will discuss the key results of the MPPG and P-SAG efforts to update and refine our understanding of the Strategic Knowledge Gaps (SKGs) required to successfully conduct human Mars missions.

Viking↗

DOE OSTI2025

Searches for direct slepton production in the compressed-mass corridor in $\sqrt{\textrm{s}}$ = 13 TeV pp collisions with the ATLAS detector

This paper presents searches for the direct pair production of charged light-flavour sleptons, each decaying into a stable neutralino and an associated Standard Model lepton. The analyses focus on the challenging ``corridor'' region, where the mass difference, $Δm$, between the slepton ($\tilde{e}$ or $\tildeμ$) and the lightest neutralino ($\tildeχ^{0}_{1}$) is less or similar to the mass of the $W$ boson, $m(W)$, with the aim to close a persistent gap in sensitivity to models with $Δm \lesssim m(W)$. Events are required to contain a high-energy jet, significant missing transverse momentum, and two same-flavour opposite-sign leptons ($e$ or $μ$). The analysis uses $pp$ collision data at $\sqrt{s} = 13$ TeV recorded by the ATLAS detector, corresponding to an integrated luminosity of 140 fb$^{-1}$. Several kinematic selections are applied, including a set of boosted decision trees. These are each optimised for different $Δm$ to provide expected sensitivity for the first time across the full $Δm$ corridor. The results are generally consistent with the Standard Model, with the most significant deviations observed with a local significance of 2.0 $σ$ in the selectron search, and 2.4 $σ$ in the smuon search. While these deviations weaken the observed exclusion reach in some parts of the signal parameter space, the previously present sensitivity gap to this corridor is largely reduced. Constraints at the 95% confidence level are set on simplified models of selectron and smuon pair production, where selectrons (smuons) with masses up to 300 (350) GeV can be excluded for $Δm$ between 2 GeV and 100 GeV.

hadron-hadron scattering↗

DOE OSTI2020

Analysis of diagonal G and subspace W approximations within fully self-consistent GW calculations for bulk semiconducting systems

Fully self-consistent GW (sc-GW) methods are now available to evaluate quasiparticle and spectral properties of various molecular and bulk systems. However, such techniques based on the full matrix of G and W are computationally demanding. Additionally, the routinely used single-shot GW approximation (G 0 W 0 ) has an undesirable dependency on the choice of initial exchange-correlation functional. In the literature, many so-called self-consistent GW methods are based on diagonal approximation of G and low-ranking approximation of W. It is thus worth checking how good such approximations are in comparison with the full matrix method. In this work, we consider AlAs, AlP, GaP, and ZnS as the prototype systems to perform sc-GW calculations by expressing the full G matrix using a plane-wave basis set. We compared our sc-GW results with the diagonal G and subspace W approximated sc-GW results (sc-GW-diagG and sc-GW-subW methods). In the sc-GW-diagG method, interacting G is expanded in the eigenvectors of noninteracting G such that only diagonal elements are retained, whereas the number of eigenmodes is truncated in sc-GW-subW calculations. A systematic analysis of the results obtained from the above techniques is presented. The differences in the quasiparticle band gap between the approximated and the full matrix sc-GW approaches are mostly less than 1.7%, which validates such widely adopted approximations, and also shows how such low-ranking approximation can be used to include higher-order terms such as the vertex correction.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2026

Lattice-Mismatched van der Waals Epitaxy and Photoluminescence of Two-Dimensional GaxIn1-xSe Alloys on Si(111)

GaxIn1-xSe (GIS) alloys are two-dimensional (2D) layered materials with band gaps and lattice parameters of interest for many energy and electronic applications. They can be fabricated using van der Waals epitaxy, which is an emerging technique that offers unprecedented opportunities for 2D optoelectronic devices and epitaxy processes. This work has demonstrated van der Waals epitaxy of GIS alloys for the first time. Films with x compositions of 0, 0.062, 0.164, 0.680, 0.894, and 1 and tunable lattice constants were grown on Si(111) substrates, and characterized by X-ray diffraction pole figure and transmission electron microscope analysis. In spite of the lattice mismatches (InSe is 4.1% too large and GaSe is 2.8% too small), these alloys grow epitaxially, with Si(111) || GIS(001) and Si[1-10] || GIS[100] orientation. Photoluminescence was used to measure tunable band gaps in the absorber-relevant 1.3-2.0 eV range as a function of x composition and showed GIS did not degrade after capping with Se and prolonged storage. Therefore, GIS alloys exhibit a technologically advantageous combination of tunable band gap and photoluminescence with relaxed lattice parameter and rotational registry with the substrate.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Modeling Data Flows with Network Calculus in Cyber-Physical Systems: Enabling Feature Analysis for Anomaly Detection Applications

The electric grid is becoming increasingly cyber-physical with the addition of smart technologies, new communication interfaces, and automated grid-support functions. Because of this, it is no longer sufficient to only study the physical system dynamics, but the cyber system must also be monitored as well to examine cyber-physical interactions and effects on the overall system. To address this gap for both operational and security needs, cyber-physical situational awareness is needed to monitor the system to detect any faults or malicious activity. Techniques and models to understand the physical system (the power system operation) exist, but methods to study the cyber system are needed, which can assist in understanding how the network traffic and changes to network conditions affect applications such as data analysis, intrusion detection systems (IDS), and anomaly detection. In this paper, we examine and develop models of data flows in communication networks of cyber-physical systems (CPSs) and explore how network calculus can be utilized to develop those models for CPSs, with a focus on anomaly and intrusion detection. This provides a foundation for methods to examine how changes to behavior in the CPS can be modeled and for investigating cyber effects in CPSs in anomaly detection applications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the order of 10\textsuperscript{5}, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but the alternative of training only on simulated data results in poor performance on real data. Domain Adaptation may be able to bridge the gap between simulated and real datasets. We utilize domain adaptation for the estimation of Einstein radius ($\Theta_E$) in simulated galaxy-scale gravitational lensing images with different levels of observational realism. We evaluate two domain adaptation techniques - Domain Adversarial Neural Networks (DANN) and Maximum Mean Discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain of lenses simulated to emulate noise conditions in the Dark Energy Survey (DES). We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain dataset. This work is the first application of domain adaptation for a regression task in strong lensing imaging analysis. Our results show the potential of using domain adaptation to perform analysis of future survey data with a deep neural network trained on simulated data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the order of $10^5$, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but the alternative of training only on simulated data results in poor performance on real data. Domain Adaptation may be able to bridge the gap between simulated and real datasets. We utilize domain adaptation for the estimation of Einstein radius ($\Theta_E$) in simulated galaxy-scale gravitational lensing images with different levels of observational realism. We evaluate two domain adaptation techniques - Domain Adversarial Neural Networks (DANN) and Maximum Mean Discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain of lenses simulated to emulate noise conditions in the Dark Energy Survey (DES). We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain dataset. This work is the first application of domain adaptation for a regression task in strong lensing imaging analysis. Our results show the potential of using domain adaptation to perform analysis of future survey data with a deep neural network trained on simulated data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

Recent advances in combustion science related to hydrogen safety

Hydrogen is a key pillar in the global Net Zero strategy. Rapid scaling up of hydrogen production, transport, distribution and utilization is expected. This entails that hydrogen, which is traditionally an industrial gas, will come into proximity of populated urban areas and in some situations handled by the untrained public. To realize all their benefits, hydrogen and its technologies must be safely developed and deployed. The specific properties of hydrogen involving wide flammability range, low ignition energy and fast flame speed implies that any accidental release of hydrogen can be easily ignited. Comparing with conventional fuels, combustion systems fueled by hydrogen are also more prone to flame instability and abnormal combustion. This paper aims to provide a comprehensive review about combustion research related to hydrogen safety. It starts with a brief introduction which includes some overview about risk analysis, codes and standards. The core content covers ignition, fire, explosions and deflagration to detonation transition (DDT). Considering that DDT leads to detonation, and that detonation may also be induced directly under special circumstances, the subject of detonation is also included for completeness. The review covers laboratory, medium and large-scale experiments, as well as theoretical analysis and numerical simulation results. While highlights are provided at the end of each section, the paper closes with some concluding remarks highlighting the achievements and key knowledge gaps.

08 HYDROGEN↗

DOE OSTI2025

Anomalous elastic softening in ferroelectric hafnia under pressure

his study employs first-principles density-functional theory (DFT) calculations to explore the elastic and mechanical properties of ferroelectric hafnia (HfO 2 ) in its polar orthorhombic 𝑃⁢𝑐⁢𝑎⁢2 1 phase under varying hydrostatic pressure conditions up to 30 GPa. Utilizing a plane-wave basis set and Perdew-Burke-Ernzerhof generalized-gradient approximation for solids in our DFT calculations, we investigate both pure and yttrium-substituted HfO 2 . Our findings reveal an anomalous reduction in the 𝐶 33 component of the elastic tensor with increasing pressure, which becomes significant above 15 GPa and signals a potential pressure-driven structural phase transition at higher pressure. The analysis of atomic displacements under pressure sheds light on the unusual mechanical behavior and phase stability of this material. Additionally, we observe a transition from an indirect band gap to a direct band gap with increasing pressure, which could have significant implications for optical applications. Here, the effects of yttrium substitution on the mechanical and electronic properties are further examined, revealing that yttrium substitution softens the elastic response of this material and reduces the electronic band gap. These results enhance our understanding of elastic and mechanical responses of ferroelectric hafnia and its potential for applications in microelectronics, piezoelectric devices, and nonvolatile ferroelectric random-access memories. Further experimental validation is recommended to confirm our predictions and explore the practical implications of the observed phase transitions and electronic behavior of the ferroelectric hafnia under high-pressure conditions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2020

An Assessment of the Usability of Machine Learning Based Tools for the Security Operations Center

Gartner, a large research and advisory company, anticipates that by 2024 80% of security operation centers (SOCs) will use machine learning (ML) based solutions to enhance their operations. 1 1 https://www.ciodive.com/news/how-data-science-tools-can-lighten-the-load-for-cybersecurity-teams/572209/ In light of such widespread adoption, it is vital for the research community to identify and address usability concerns. This work presents the results of the first in situ usability assessment of ML-based tools. With the support of the US Navy, we leveraged the national cyber range-a large, air-gapped cyber testbed equipped with state-of-the-art network and user emulation capabilities-to study six US Naval SOC analysts' usage of two tools. Our analysis identified several serious usability issues, including multiple violations of established usability heuristics for user interface design. We also discovered that analysts lacked a clear mental model of how these tools generate scores, resulting in mistrust a and/or misuse of the tools themselves. Surprisingly, we found no correlation between analysts' level of education or years of experience and their performance with either tool, suggesting that other factors such as prior background knowledge or personality play a significant role in ML-based tool usage. Our findings demonstrate that ML-based security tool vendors must put a renewed focus on working with analysts, both experienced and inexperienced, to ensure that their systems are usable and useful in real-world security operations settings.

Oesch, T↗