Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

SSHAC Level 1 Probabilistic Seismic Hazard Analysis for the Idaho National Laboratory

A Probabilistic Seismic Hazard Analysis (PSHA) was completed for the Materials and Fuels Complex (MFC), Naval Reactors Facility (NRF), and the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL) (Figure 1-1). The PSHA followed the approaches and procedures appropriate for a Study Level 1 provided in the guidance advanced by the Senior Seismic Hazard Analysis Committee (SSHAC) in U.S. Nuclear Regulatory Commission (NRC) NUREG/CR-6372 and NUREG-2117 (NRC, 1997; 2012a). The SSHAC Level 1 PSHAs for MFC and ATR were conducted as part of the Seismic Risk Assessment (SRA) project (INL Project number 31287) to develop and apply a new-risk informed methodology, respectively. The SSHAC Level 1 PSHA was conducted for NRF to provide guidance on the potential use of a design margin above rock hazard levels. The SRA project is developing a new risk-informed methodology that will provide a systematic approach for evaluating the need for an update of an existing PSHA. The new methodology proposes criteria to be employed at specific analysis, decision, or comparison points in its evaluation process. The first four of seven criteria address changes in inputs and results of the PSHA and are given in U.S. Department of Energy (DOE) Standard, DOE-STD-1020-2012 (DOE, 2012a) and American National Standards Institute/American Nuclear Society (ANSI/ANS) 2.29 (ANS, 2008a). The last three criteria address evaluation of quantitative hazard and risk-focused information of an existing nuclear facility. The seven criteria and decision points are applied to Seismic Design Category (SDC) 3, 4, and 5, which are defined in American Society of Civil Engineers/Structural Engineers Institute (ASCE/SEI) 43-05 (ASCE, 2005). The application of the criteria and decision points could lead to an update or could determine that such update is not necessary.

58 GEOSCIENCES↗

NASA Johnson Space Center: Total quality partnership

The development of and benefits realized from a joint NASA, support contractor continuous improvement process at the Johnson Space Center (JSC) is traced. The joint effort described is the Safety, Reliability, and Quality Assurance Directorate relationship with its three support contractors which began in early 1990. The Continuous Improvement effort started in early 1990 with an initiative to document and simplify numerous engineering change evaluation processes. This effort quickly grew in scope and intensity to include process improvement teams, improvement methodologies, awareness, and training. By early 1991, the support contractor had teams in place and functioning, program goals established and a cultural change effort underway. In mid-l991 it became apparent that a major redirection was needed to counter a growing sense of frustration and dissatisfaction from teams and managers. Sources of frustration were isolated to insufficient joint participation on teams, and to a poorly defined vision. Over the next year, the effort was transformed to a truly joint process. The presentation covers the steps taken to define vision, values, goals, and priorities and to form a joint Steering Committee and joint process improvement teams. The most recent assessment against the President's award criteria is presented as a summary of progress. Small, but important improvement results have already demonstrated the value of the joint effort.

Harlan, Charlie↗

SU-E-T-510: Interplay Between Spots Sizes, Spot / Line Spacing and Motion in Spot Scanning Proton Therapy

Purpose In proton beam configuration for spot scanning proton therapy (SSPT), one can define the spacing between spots and lines of scanning as a ratio of given spot size. If the spacing increases, the number of spots decreases which can potentially decrease scan time, and so can whole treatment time, and vice versa. However, if the spacing is too large, the uniformity of scanned field decreases. Also, the field uniformity can be affected by motion during SSPT beam delivery. In the present study, the interplay between spot/ line spacing and motion is investigated. Methods We used four Gaussian-shape spot sizes with 0.5cm, 1.0cm, 1.5cm, and 2.0cm FWHM, three spot/line spacing that creates uniform field profile which are 1/3*FWHM, σ/3*FWHM and 2/3*FWHM, and three random motion amplitudes within, +/−0.3mm, +/−0.5mm, and +/−1.0mm. We planned with 2Gy uniform single layer of 10×10cm2 and 20×20cm2 fields. Then, mean dose within 80% area of given field size, contrubuting MU per each spot assuming 1cGy/MU calibration for all spot sizes, number of spots and uniformity were calculated. Results The plans with spot/line spacing equal to or smaller than 2/3*FWHM without motion create ∼100% uniformity. However, it was found that the uniformity decreases with increased spacing, and it is more pronounced with smaller spot sizes, but is not affected by scanned field sizes. Conclusion It was found that the motion during proton beam delivery can alter the dose uniformity and the amount of alteration changes with spot size which changes with energy and spot/line spacing. Currently, robust evaluation in TPS (e.g. Eclipse system) performs range uncertainty evaluation using isocenter shift and CT calibration error. Based on presented study, it is recommended to add interplay effect evaluation to robust evaluation process. For future study, the additional interplay between the energy layers and motion is expected to present volumetric effect.

60 APPLIED LIFE SCIENCES↗

Stochastic process approximation for recursive estimation with guaranteed bound on the error covariance

An approach, is proposed for the design of approximate, fixed order, discrete time realizations of stochastic processes from the output covariance over a finite time interval, was proposed. No restrictive assumptions are imposed on the process; it can be nonstationary and lead to a high dimension realization. Classes of fixed order models are defined, having the joint covariance matrix of the combined vector of the outputs in the interval of definition greater or equal than the process covariance; (the difference matrix is nonnegative definite). The design is achieved by minimizing, in one of those classes, a measure of the approximation between the model and the process evaluated by the trace of the difference of the respective covariance matrices. Models belonging to these classes have the notable property that, under the same measurement system and estimator structure, the output estimation error covariance matrix computed on the model is an upper bound of the corresponding covariance on the real process. An application of the approach is illustrated by the modeling of random meteorological wind profiles from the statistical analysis of historical data.

Menga, G.↗

Nuclear Data Sheets for A=203

Evaluated nuclear structure and decay data for all nuclei with mass number A=203 ( 203 Os, 203 Ir, 203 Pt, 203 Au, 203 Hg, 203 Tl, 203 Pb, 203 Bi, 203 Po, 203 At, 203 Rn, 203 Fr, 203 Ra) are presented. All available experimental data are compiled and evaluated, and best values for level and γ-ray energies, quantum numbers, lifetimes, γ-ray intensities and transition probabilities, as well as other nuclear properties, are recommended. Inconsistencies and discrepancies that exist in the literature are discussed. A number of computer codes (https://www-nds.iaea.org/public/ensdf_pgm/index.htm) developed by members of the NSDD network were used during the evaluation process. For example, the reported absolute γ-ray emission probabilities and their uncertainties were determined using the GABS code. Here, the γ-ray transition probabilities were determined using the RULER code and the corresponding uncertainties were estimated by means of a Monte-Carlo approach. This work supersedes the earlier evaluation by F.G. Kondev (2005Ko20), published in Nuclear Data Sheets 105, 1 (2005).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

An Exploration of Mercury Soils Treatment Technologies for the Y-12 Plant - 13217

There are a number of areas at the Y-12 National Security Complex (Y-12) that have been contaminated with mercury due to historical mercury use and storage. Remediation of these areas is expected to generate large volumes of waste that are Resource Conservation and Recovery Act (RCRA) characteristically hazardous. These soils will require treatment to meet RCRA Land Disposal Restrictions (LDR) prior to disposal. URS - CH2M Oak Ridge LLC (UCOR) performed a feasibility assessment to evaluate on-site and off-site options for the treatment and disposal of mercury-contaminated soil from the Y-12 Site. The focus of the feasibility assessment was on treatment for disposal at the Environmental Management Waste Management Facility (EMWMF) located on the Oak Ridge Reservation. A two-phase approach was used in the evaluation process of treatment technologies. Phase 1 involved the selection of three vendors to perform treatability studies using their stabilization treatment technology on actual Y-12 soil. Phase II involved a team of waste management specialists performing an in-depth literature review of all available treatment technologies for treating mercury contaminated soil using the following evaluation criteria: effectiveness, feasibility of implementation, and cost. The result of the treatability study and the literature review revealed several viable on-site and off-site treatment options. This paper presents the methodology used by the team in the evaluation of technologies especially as related to EMWMF waste acceptance criteria, the results of the physical treatability studies, and a regulatory analysis for obtaining regulator approval for the treatment/disposal at the EMWMF. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Nuclear Data Sheets for A=201

Evaluated nuclear structure and decay data for all nuclei with mass number A=201 ( 201 Os, 201 Ir, 201 Pt, 201 Au, 201 Hg, 201 Tl, 201 Pb, 201 Bi, 201 Po, 201 At, 201 Rn, 201 Fr, 201 Ra) are presented. All available experimental data are compiled and evaluated, and best values for level and γ-ray energies, quantum numbers, lifetimes, γ-ray intensities and transition probabilities, as well as other nuclear properties, are recommended. Inconsistencies and discrepancies that exist in the literature are discussed. A number of computer codes (https://wwwnds.iaea.org/public/ensdf pgm/index.htm) developed by members of the NSDD network were used during the evaluation process. For example, the reported absolute γ-ray emission probabilities and their uncertainties in various decay data sets were determined using the GABS code. The γ-ray transition probabilities were determined using the RULER code and the corresponding uncertainties were determined using a Monte-Carlo approach. In conclusion, this work supersedes the earlier evaluation by F.G. Kondev (2007Ko06), published in Nuclear Data Sheets 108, 365 (2007).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Lignin-based jet fuel and its blending effect with conventional jet fuel

Sustainable aviation fuels (SAFs) must demonstrate specific physical and chemical properties as well as material compatibility (i.e., seal swell) to be used as an aviation turbine fuel. Several alternative jet fuels incorporated in ASTM D7566 are comprised mainly of n/iso-alkanes and can only be blended up to 50 vol% due to material compatibility and density issues. Prior work illustrated the ability of cycloalkanes to replace aromatics’s role in material compatibility. Here, we report the first archival documentation of a feedstock and chemical process to yield a product composition able to complement existing SAF ASTM D7566 annexes. A lignin-based jet fuel (LJF) blend components is generated and composed of mostly C7-C18 mono, di, and tri-cycloalkanes. The neat LJF was blended with conventional jet fuel at 10 vol% (LJF blend) to simulate an ASTM “Fast Track” evaluation process. Fuel properties that are critical to engine operability (ATSM D4054 Tier 3 & 4) were either predicted or experimental tested based on the volume availability. All LJF blend’s critical properties fall within the experience range of conventional jet fuel, with o-ring swelling exceeding the typical range of conventional fuels. Here, these results in total support the potential use of this LJF pathway to complement other SAF pathways and achieve 100% drop-in SAF.

09 BIOMASS FUELS↗

Thermal properties and Thermodynamic Equilibrium Modeling of Cementitious Waste Forms

INTRODUCTION Cementitious matrices are used in the US DOE complex and worldwide to solidify aqueous radioactive, hazardous, mixed salt solutions, and sludges to meet low-level radioactive waste (LLW) disposal requirements. Blended formulations are used for most waste form mix designs. Substitution of pozzolans for Class F fly ash has the potential to alter the processing properties and stabilization properties of the resulting waste forms because they add chemical and mineralogical complexity to the final material. The Savannah River Site saltstone waste form was selected as the test case material. CemGEMS software was chosen as the model for predicting hydrated cementitious phases assemblages and phase evolution. Isothermal calorimetry was used as the method for evaluating processing and properties of fresh, uncured waste forms. The test cases consisted of reference case saltstone, and five natural pozzolan substituted saltstone mixes.

Bustamante, Michael E. [Savannah River National La↗

Use-Inspired, Process-Oriented GCM Selection: Prioritizing Models for Regional Dynamical Downscaling

Dynamical downscaling is a crucial process for providing regional climate information for broad uses, using coarser-resolution global models to drive higher-resolution regional climate simulations. The pool of global climate models (GCMs) providing the fields needed for dynamical downscaling has increased from the previous generations of the Coupled Model Intercomparison Project (CMIP). However, with limited computational resources, the need for prioritizing the GCMs for subsequent downscaling studies remains. GCM selection for dynamical downscaling should focus on evaluating processes relevant for providing boundary conditions to the regional models and be inspired by regional uses such as the response of extremes to changes in the boundary conditions. This leads to the need for metrics representing processes of relevance to diverse stakeholders and subregions of a domain. Procedures to account for metric redundancy and the statistical distinguishability of GCM rankings are required. Further, procedures for selecting realizations from ensembles of top-performing GCM simulations can be used to span the range of climate change signals in multiple ways. As a result, distinct weighting of metrics and prioritization of particular realizations may depend on user needs. We provide high-level guidelines for such region-specific evaluations and address how CMIP7 might enable dynamical downscaling of a representative sample of high-quality models across representative shared socioeconomic pathways (SSPs).

54 ENVIRONMENTAL SCIENCES↗

Vapor Extraction Well Performance and Recommendations for Future Soil Vapor Extraction Activities at the A-014 Outfall

The vadose zone near the A-014 Outfall contains residual volatile organic compounds that serve as a potential source for groundwater contamination. Since 1995 active soil vapor extraction has been an integral part of the corrective actions at this facility. Currently two separate soil vapor extraction systems are being used to remove contamination in the vapor phase from the subsurface. The 782-3M soil vapor extraction unit was deployed in April of 1995 and the A-014 Mobile #3 unit was deployed in January of 2009. The objective of this investigation is to evaluate the performance of individual wells associated with these systems and provide recommendations on possible optimizations to these systems. Options that are considered include continuation of active extraction, transition to a low-energy soil vapor extraction treatment, or abandon nonperforming wells. The scope of the investigation examined the eighteen (18) vadose zone wells that are associated with the ongoing Resource Conservation and Recovery Act corrective action program within A/M Area. During the evaluation process, individual wells were isolated and purged for a standard period using the existing soil vapor extraction system. Following the purge period, vapor samples were collected intermittently over the following 24–72-hour period. The objective of the sampling activity was to collect sufficient data to estimate the mass flow rate of contaminants from the individual extraction wells. Subsequent analysis then evaluated and ranked extraction performance to provide recommendations on the viability, operational scheme, and applicability of low energy soil vapor extraction.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Recent Efforts in Data Compilations for Nuclear Astrophysics

Some recent efforts in compiling data for astrophysical purposes are introduced, which were discussed during a JINA-CARINA Collaboration meeting on 'Nuclear Physics Data Compilation for Nucleosynthesis Modeling' held at the ECT* in Trento/Italy from May 29th-June 3rd, 2007. The main goal of this collaboration is to develop an updated and unified nuclear reaction database for modeling a wide variety of stellar nucleosynthesis scenarios. Presently a large number of different reaction libraries (REACLIB) are used by the astrophysics community. The 'JINA Reaclib Database' on http://www.nscl.msu.edu/{approx}nero/db/ aims to merge and fit the latest experimental stellar cross sections and reaction rate data of various compilations, e.g. NACRE and its extension for Big Bang nucleosynthesis, Caughlan and Fowler, Iliadis et al., and KADoNiS.The KADoNiS (Karlsruhe Astrophysical Database of Nucleosynthesis in Stars, http://nuclear-astrophysics.fzk.de/kadonis) project is an online database for neutron capture cross sections relevant to the s process. The present version v0.2 is already included in a REACLIB file from Basel university (http://download.nucastro.org/astro/reaclib). The present status of experimental stellar (n,{gamma}) cross sections in KADoNiS is shown. It contains recommended cross sections for 355 isotopes between {sup 1}H and {sup 210}Bi, over 80% of them deduced from experimental data.A ''high priority list'' for measurements and evaluations for light charged-particle reactions set up by the JINA-CARINA collaboration is presented. The central web access point to submit and evaluate new data is provided by the Oak Ridge group via the http://www.nucastrodata.org homepage. 'Workflow tools' aim to make the evaluation process transparent and allow users to follow the progress.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Short-cut Methods versus Rigorous Methods for Performance-evaluation of Distillation Configurations

Here, this study demonstrates the efficacy of a short-cut method such as the Global Minimization Algorithm (GMA), that uses assumptions of ideal mixtures, constant molar overflow (CMO) and pinched columns, in pruning the search-space of distillation column configurations for zeotropic multicomponent separation, to provide a small subset of attractive configurations with low minimum heat duties. The short-cut method, due to its simplifying assumptions, is computationally efficient, yet reliable in identifying the small subset of useful configurations for further detailed process evaluation. This two-tier approach allows expedient search of the configuration space containing hundreds to thousands of candidate configurations for a given application.

42 ENGINEERING↗

Experiences with SYCL on AMD GPUs with Kokkos

With the recent diversification of the hardware landscape in the high-performance computing (HPC) community, performance-portability solutions are becoming more and more important. One of the most popular choices is Kokkos, which recently became a Linux Foundation project. Most of its development is supported by the US Department of Energy and the French Alternative Energies and Atomic Energy Commission. Kokkos is implemented as a C++ library with multiple backends to support CPUs as well as various GPU architectures. These backends include OpenMP, CUDA, HIP, and also SCYL. This approach enables users to leverage the preferred vendor toolchain for the respective platform (e.g. CUDA, ROCm, OneAPI). The SYCL backend is used to target Intel GPUs, in particular to support the Aurora exascale supercomputer. However, SYCL itself also offers a large degree of portability, and in fact Kokkos’ CI for SYCL has been running on NVIDIA hardware due to a lack of access to Intel GPUs. In this report, we describe our experience with using Kokkos SYCL backend on AMD GPUs targeting the Frontier supercomputer at Oak Ridge National Laboratory. The two major SYCL implementations are DPC++ and AdaptiveCpp. While the Kokkos SYCL backend has been implemented using the former, the latter was the first implementation to target AMD GPUs. We will discuss the experience with both of these SYCL implementations in terms of functionality and performance. Using Kokkos to evaluate SYCL toolchains has a number of benefits. Kokkos’ use of SYCL is fairly complex, exercising features such as graphs, relocatable device functions, atomics – including for non-arithmetic types, as well as pinned and page migratable memory allocations. Kokkos also needs to implement capabilities such as Kokkos’ hierarchical parallelism that are not a straight-forward mapping to SYCL capabilities. Furthermore, a large number of libraries and applications that represent diverse use cases are implemented in Kokkos, providing readily available test cases for a toolchain evaluation. Preliminary results show that support for AMD GPUs in DPC++ is much less mature than for NVIDIA GPUs or Intel GPUs. While the situation has improved significantly over the last year, we still encounter many runtime failures, dispatching problems, and code generation issues. With AdaptiveCpp the challenges arise even earlier in the evaluation process. Since Kokkos’ SYCL implementation is largely focused on supporting Intel GPUs, we opted to leverage SYCL extensions which are available in DPC++ but not in AdaptiveCpp. Furthermore, AdaptiveCpp appears to be less conformant with the SYCL2020 standard which Kokkos relies on. In some cases, we are able to work around the lack of feature support, in other cases we have to disable certain Kokkos capabilities to evaluate the toolchain. Our evaluation will leverage Kokkos’ unit tests to establish basic functionality and feature completeness. We then use simple benchmarks for components of a CG implementation as a measure of usability and performance of the SYCL toolchains.

97 MATHEMATICS AND COMPUTING↗

Evaluation of Low Dose Silicon Carbide Temperature Monitors

Thermocouples are generally used to provide real-time temperature indications in instrumented tests performed at materials and test reactors. Melt wires or paint spots are often included in such tests as an independent technique of detecting peak temperatures incurred during irradiation. In addition, less expensive static capsule tests, which have no leads attached for real-time data transmission, often rely on melt wires and paint spots as a post-irradiation technique for peak temperature indication. Unfortunately, these techniques are limited in that they can only detect whether a single temperature is or is not exceeded. Silicon carbide (SiC) monitors are advantageous because a single monitor can be used to determine the peak temperature reached within a relatively broad range (200 – 800°C). Although the use of SiC monitors was proposed more than five decades ago, the ultimate performance limits of this technique are not fully understood. The Nuclear Science User Facilities (NSUF) is the United States Department of Energy Office of Nuclear Energy's only designated nuclear energy user facility. Its mission is to provide nuclear energy researchers access to world-class capabilities and to facilitate the advancement of nuclear science and technology. This mission is supported by providing access to state-of-the-art experimental irradiation testing, post irradiation examination facilities, and high performance computing capabilities as well as technical and scientific assistance for the design and execution of projects. As part of an NSUF project, low dose silicon carbide monitors were irradiated in the Belgian Reactor 2 and were then evaluated both at the SCK•CEN and at Idaho National Laboratory's High Temperature Test Laboratory to determine their peak temperature achieved during irradiation. The technical significance of this work was that the monitors were irradiated to a dose that was significantly less than recommended in published literature. This paper will discuss the evaluation process, the irradiation test, and the performance of the low dose silicon carbide temperature monitors.

36 MATERIALS SCIENCE↗

Transforming any Facility for Meeting Strict Cleanliness Requirements

Over the life of the James Webb Space Telescope (JWST), Integration & Test (I&T) has taken place in areas that needed considerable work to make the facility itself and/or the protocols used while working in the rooms suitable to meet JWST percent area coverage (PAC) and molecular accumulation requirements. In addition to normal particulate matter, JWST had a uniquely significant challenge: fibers! Fibers not only cause much higher PAC levels, but they also risk damaging the angstrom sized Near Infrared Spectrometer (NIRSpec) microshutter array (MSA), which is critical to NIRSpec instrument performance. The primary emphasis of this paper is to address particulate and fiber contamination. The success of the JWST mission required effective cleanrooms, protocols, and mitigations in non-cleanroom areas that were pressed into service to house contamination-sensitive optics and scientific instruments. Some presented profound challenges. These included: NASA’s 60-year-old Johnson Space Center (JSC) Chamber A, which had never been used for anything contamination-sensitive, and the European tropical launch facilities, which were designed to meet International Standard Organization (ISO) Class 8 processing for communication satellites. The final challenge for JWST, as if to stare us in the face and say, “I dare you to try and make me clean enough,” was preparing the 4 areas in the Centre Spatial Guyanais (CSG) Final Assembly Building (BAF) located in French Guiana, a building in which one entire side opens for Ariane 5 rocket ingress and egress. This paper will describe our initial evaluation processes and the actual work undertaken to transform even the most challenging areas into first class cleanrooms that met JWST particulate and fiber requirements.

James Webb Space Telescope↗

Design of a welded joint for robotic, on-orbit assembly of space trusses

In the future, some spacecraft will be so large that they must be assembled on-orbit. These spacecraft will be used for such tasks as manned missions to Mars or used as orbiting platforms for monitoring the Earth or observing the universe. Some large spacecraft will probably consist of planar truss structures to which will be attached special purpose, self-contained modules. The modules will most likely be taken to orbit fully outfitted and ready for use in heavy-lift launch vehicles. The truss members will also similarly be taken to orbit, but most unassembled. The truss structures will need to be assembled robotically because of the high costs and risks of extra-vehicular activities. Some missions will involve very large loads. To date, very few structures of any kind have been constructed in space. Two relatively simple trusses were assembled in the Space Shuttle bay in late 1985. Here the development of a design of a welded joint for on-orbit, robotic truss assembly is described. Mechanical joints for this application have been considered previously. Welded joints have the advantage of allowing the truss members to carry fluids for active cooling or other purposes. In addition, welded joints can be made more efficient structurally than mechanical joints. Also, welded joints require little maintenance (will not shake loose), and have no slop which would cause the structure to shudder under load reversal. The disadvantages of welded joints are that a more sophisticated assembly robot is required, weld flaws may be difficult to detect on-orbit, the welding process is hazardous, and welding introduces contamination to the environment. In addition, welded joints provide less structural damping than do mechanical joints. Welding on-orbit was first investigated aboard a Soyuz-6 mission in 1969 and then during a Skylab electron beam welding experiment in 1973. A hand held electron beam welding apparatus is currently being prepared for use on the MIR space station. Presently, Marshall Space Flight Center is evaluating processes appropriate for on-orbit welding. A low gravity environment has been found to have very minor effects on the welding processes appropriate for this application. This is based on tests run on-orbit as well as low gravity environments achieved by flying aircraft in parabolic trajectories. It appears that a modified form of gas tungsten arc welding (GTAW) will be most appropriate for welding together structures on-oribt. The process has been modified to work in a vacuum by providing gas to the arc zone by means of a hollow tungsten electrode with special shielding. A commercial tube welding head has been successfully modified for use on-orbit with a gas leakage rate of approximately 2.5 liters/min. To develop as realistic a joint as possible, a specific truss structure was selected on which to base the design. The structure considered was based on the 120 foot diameter aerobrake tetrahedral truss structure. The truss members were assumed to consist of graphite/epoxy tubes. Also, it was assumed that the nodes were constructed of 2219-T87 aluminum alloy. The magnitude of the member load assumed for design purposes was 100 kips.

Rule, William K.↗

AI Model Benchmarking for Nonproliferation Applications: Steel Thread Benchmarking Task Force Technical Report (Rev. 2)

Steel Thread is a NA-22 venture that seeks to build trustworthy, reliable AI models that can be used in a wide variety of nonproliferation tasks. A key aspect of building these models is developing appropriate benchmarks and evaluation methods, which will enable the venture to identify and adapt models to provide the most value in the nonproliferation domain. Benchmarks must be relevant to key tasks in this domain, such as question answering, information retrieval, document summarization and classification, consensus analysis, and image and data analysis. This report 1) provides an overview of benchmark design, evaluation, and challenges; 2) reviews a variety of open benchmarks, with a focus on language models and tasks; and 3) identifies benchmarks that are most relevant to Steel Thread. This report is intended to serve as a basis for further efforts to classify and evaluate benchmarks and their correlation with success on nonproliferation-specific tasks. The Steel Thread venture has defined benchmarks to be a particular combination of a dataset (or datasets) and a metric (or metrics) conceptualized as representing one or more specific tasks or sets of abilities for a specific modality. It is adopted by a research community as a shared framework for comparing methods.1 It includes 1) Data: Labeled (a designated subset not used for training, which could be all the data), 2) Metric: A way to quantify performance, 3) Task/Ability: What the benchmark is testing, 4) Protocol: A structured and repeatable evaluation process, 5) Baseline/Reference Model: For comparison; could be statistical, rule-based, SME-derived, or another model, and 6) Maintenance Plan: to update with new information over time; important for long-term utility. For further clarity, the definition includes what a benchmark, in this context, is not. It is not a corpus of training data, specific to a model (it is intended to apply to a range of models), a universal evaluation of performance, a guarantee that the ‘top’ model on the leaderboard will be the best fit for every specific use case, an all-encompassing proof of a model’s universal quality, nor is it a one-size-fits-all measure of success. It does not cover every real-world constraint (like operational, ethical, or cost considerations), a systems integration test, or a unit test. This definition was inspired by and resulted from discussions within the Steel Thread Benchmarking Task Force. This group was formed to define what we would mean as a benchmark within Steel Thread but persisted as the need to develop a thorough understanding of the large and expanding existing benchmarking space. This technical report is a result of the group’s divide and conquer approach to exploring this space. The release of benchmarks might not be progressing as quickly as model development, but it is moving very fast, as many benchmarks quickly become saturated, when state-of-the-art models score so close to the benchmark’s ceiling that their results are virtually indistinguishable. At that point, the test no longer differentiates between new systems, so researchers usually stop reporting scores as the benchmark no longer informs about improvements from the next generation of models. In the OpenAI announcement of GPT-5, they reported results on six flagship public benchmarks (AIME 2025, SWE-bench Verified, Aider Polyglot, MMMU, HealthBench Hard, GPQA) but the full system-card covers roughly thirty-five separate evaluations, comprising hundreds of test task items in total. There have been some efforts to summarize benchmarks in specific fields, like for text-to-image generation, but these surveys have had a narrow methodology scope. Therefore, a comprehensive survey of all benchmarks or even all benchmarks that could be relevant to Steel Thread is outside of the scope of this report. We chose some specific benchmarks to investigate in detail.

97 MATHEMATICS AND COMPUTING↗