Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data quality metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

SU-D-204-07: Comparison of AAPM TG150 Draft Image Receptor Tests with Vendor Automated QC Tests for Five Mobile DR Units

Purpose: To determine the consistency of digital detector (DR) tests recommended by AAPM TG150 and tests provided by commercially available DirectView Total Quality Tool (TQT). Methods: The DR tests recommended by the TG150 Detector Subgroup[1] were performed on 4 new Carestream DRX-Revolution and one Carestream DRX1C retrofit of a GE AMX-4 that had been in service for three years. After detector calibration, flat-field images plus images of two bar patterns oriented parallel and perpendicular to the A-C axis, were acquired at conditions recommended by TG150. Raw images were harvested and then analyzed using a MATLAB software previously validated[2,3,4]. Data were analyzed using ROIs of two different dimensions: 1) 128 x 128 ROIs matching the detector electronics; and 2) 256 x 256 ROIs, each including 4 adjacent smaller ROIs. TG150 metrics from 128 x 128 ROIs were compared to TQT metrics, which are also obtained from 128 x 128 ROIs[5]. Results: The results show that both TG150 and TQT measurements were consistent among these detectors. Differences between TG150 and TQT values appear systematic. Compared with 128 x 128 ROIs, noise and SNR non-uniformity were lower with 256 x 256 ROIs, although signal non-uniformity was similar, indicating detectors were appropriately calibrated for gain and offset. MTF of the retrofit unit remained essentially the same between 2012 and 2015, but was inferior to the new units. The older generator focal spot is smaller (0.75mm vs. 1.2mm), and the SID for acquisition is 182cm as well, so focal spot dimensions cannot explain the difference. The difference in MTF may be secondary to differences in generator X-ray spectrum or by unannounced changes in detector architecture. Further investigation is needed. Conclusion: The study shows that both TG150 and TQT tests are consistent. The numerical value of some metrics are dependent on ROI size.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows

This report details recent progress for the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows”. We refer to the project as IPPD/2, reflecting the 2017 renewal under expanded scope and partners In IPPD/2, we increased our research scope to include data motion. We are focusing on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. This new work on data motion will augment and complement IPPD/2’s research that focused on the computational aspects of tasks. We leverage and extend our existing tools and demonstrate our work on the Belle II workflow suite as well as on workflows from NSLS-II. The highlights of our work are as follows: Provenance for Workflows: Provenance is used to provide information enabling quality control, re-run computational workflows, and reproduce results. IPPD/2 has been building a scalable provenance management system that enables the capture of provenance from the high-level workflow through all relevant system levels in one integrated environment. Leveraging this work, our recent efforts have included using provenance as an enabling technique. Workload characterization: Leveraging provenance and analysis, we characterize data movement within network, storage, and memory over a variety of workloads. This characterization enables an understanding by performance analysts and application developers of the range of behaviors that could be expected. Performance Prediction for Workflows: The goal of modeling distributed workflows is to understand performance bottlenecks and enable more intelligent task scheduling to optimize selected metrics of interest (e.g., task throughput or output data rate). IPPD/2 has utilized both analytical and AI/ML modeling methodologies for performance modeling. Advanced Scheduling and Fault Modeling for Workflows: Scheduling of large-scale scientific workflows on geographically distributed resources is a challenging problem. To improve workflow throughput, we combined novel scheduling algorithms with task predictions from performance modeling and fault modeling. Dynamically Alleviating Bottlenecks in Workflows: Exploiting our provenance, analysis, and modeling efforts, we have explored and developed several techniques for dynamically detecting and alleviating bottlenecks in data movement. In particular, we have spent considerable effort demonstrating our techniques on production-like workflow configurations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Reliable and explainable machine-learning methods for accelerated material discovery

Abstract Despite ML’s impressive performance in commercial applications, several unique challenges exist when applying ML in materials science applications. In such a context, the contributions of this work are twofold. First, we identify common pitfalls of existing ML techniques when learning from underrepresented/imbalanced material data. Specifically, we show that with imbalanced data, standard methods for assessing quality of ML models break down and lead to misleading conclusions. Furthermore, we find that the model’s own confidence score cannot be trusted and model introspection methods (using simpler models) do not help as they result in loss of predictive performance (reliability-explainability trade-off). Second, to overcome these challenges, we propose a general-purpose explainable and reliable machine-learning framework. Specifically, we propose a generic pipeline that employs an ensemble of simpler models to reliably predict material properties. We also propose a transfer learning technique and show that the performance loss due to models’ simplicity can be overcome by exploiting correlations among different material properties. A new evaluation metric and a trust score to better quantify the confidence in the predictions are also proposed. To improve the interpretability, we add a rationale generator component to our framework which provides both model-level and decision-level explanations. Finally, we demonstrate the versatility of our technique on two applications: (1) predicting properties of crystalline compounds and (2) identifying potentially stable solar cell materials. We also point to some outstanding issues yet to be resolved for a successful application of ML in material science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

SU-F-P-11: Long Term Dosimetric Stability of 6 TomoTherapy Systems

Purpose: The dosimetric stability of six TomoTherapy units was analyzed to investigate changes in performance over time and with system upgrades. Methods: Energy and output were tracked using monitor chamber signal, onboard MVCT detector signal and external ion chamber measurements. The systems (and monitoring periods) include 3 Hi-Art (67, 61 and 65 mos.), 2 HDA (29 and 25 mos.) and one research unit (7 mo.). Dose Control Stability system (DCS) was installed on 4 systems. Output stability is reported as deviation from reference monitor chamber signal for all systems, and from an external chamber for 4 systems. Energy stability was monitored using the relative (center versus off-axis) MVCT detector signal and/or the ratio of chamber measurements at 2 depths. The results from the clinical systems were used to benchmark the stability of the research unit, which has the same linear accelerator but runs at a higher dose rate. Results: The output based on monitor chamber data of all six systems is very stable. Non- DCS had a standard deviation of 1.7% and 1.8%. As expected, DCS systems had improved standard deviation: 0.003–0.05%. The energy was also very stable for all units. The standard deviation in exit detector flatness was 0.02–0.3%. Ion chamber output and 20/10 cm ratios supported these results. The stability for the research system, as monitored with a variety of metrics, is on par with the existing systems. Conclusion: The output and energy of six TomoTherapy units over a total of almost 10 years is quite stable. For each system, the results are consistent between the different measurement tools and techniques, proving not only the dosimetric stability, but that these quality parameters can be confirmed with various metrics. A research unit operating at a higher dose rate performed as well as the clinical treatment units. University of Wisconsin and Accuray Inc. (vendor of TomoTherapy systems) have a research agreement which supplies funds for research to the University. This project was partially supporting with these funds.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

A Causal Inference Model Based on Random Forests to Identify the Effect of Soil Moisture on Precipitation

Soil moisture influences precipitation mainly through its impact on land–atmosphere interactions. Understanding and correctly modeling soil moisture–precipitation (SM–P) coupling is crucial for improving weather forecasting and subseasonal to seasonal climate predictions, especially when predicting the persistence and magnitude of drought. However, the sign and spatial structure of SM–P feedback are still being debated in the climate research community, mainly due to the difficulty in establishing causal relationships and the high degree of nonlinearity in land–atmosphere processes. To this end, we developed a causal inference model based on the Granger causality analysis and a nonlinear machine learning model. This model includes three steps: nonlinear anomaly decomposition, nonlinear Granger causality analysis, and evaluation of the quality of SM–P feedback, which eliminates the nonlinear response of interannual and seasonal variability and the memory effects of climatic factors and isolates the causal relationship of local SM–P feedback. We applied this model by using National Climate Assessment–Land Data Assimilation System (NCA-LDAS) datasets over the United States. Here, the results highlight the importance of nonlinear atmosphere responses in land–atmosphere interactions. In addition, the strong feedback over the southwestern United States and the Great Plains both highlight the impacts of topographic factors rather than only the sensitivity of evapotranspiration to soil moisture. Furthermore, the SM–P index defined by our framework is used to benchmark Earth system models (ESMs), which provides a new metric for efficiently identifying potential model biases in modeling local land–atmosphere interactions and may help the development of ESMs in improving simulations of water cycle variability and extremes.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2011

CCSDS Advanced Orbiting Systems Virtual Channel Access Service for QoS MACHETE Model

To support various communications requirements imposed by different missions, interplanetary communication protocols need to be designed, validated, and evaluated carefully. Multimission Advanced Communications Hybrid Environment for Test and Evaluation (MACHETE), described in "Simulator of Space Communication Networks" (NPO-41373), NASA Tech Briefs, Vol. 29, No. 8 (August 2005), p. 44, combines various tools for simulation and performance analysis of space networks. The MACHETE environment supports orbital analysis, link budget analysis, communications network simulations, and hardware-in-the-loop testing. By building abstract behavioral models of network protocols, one can validate performance after identifying the appropriate metrics of interest. The innovators have extended the MACHETE model library to include a generic link-layer Virtual Channel (VC) model supporting quality-of-service (QoS) controls based on IP streams. The main purpose of this generic Virtual Channel model addition was to interface fine-grain flow-based QoS (quality of service) between the network and MAC layers of the QualNet simulator, a commercial component of MACHETE. This software model adds the capability of mapping IP streams, based on header fields, to virtual channel numbers, allowing extended QoS handling at link layer. This feature further refines the QoS v existing at the network layer. QoS at the network layer (e.g. diffserv) supports few QoS classes, so data from one class will be aggregated together; differentiating between flows internal to a class/priority is not supported. By adding QoS classification capability between network and MAC layers through VC, one maps multiple VCs onto the same physical link. Users then specify different VC weights, and different queuing and scheduling policies at the link layer. This VC model supports system performance analysis of various virtual channel link-layer QoS queuing schemes independent of the network-layer QoS systems.

Jennings, Esther H.↗
DOE OSTI2022

Distributionally Robust Decision Making Leveraging Conditional Distributions

Distributionally robust optimization (DRO) is a powerful tool for decision making under uncertainty. It is particularly appealing because of its ability to leverage existing data. However, many practical problems call for decision-making with some auxiliary information, and DRO in the context of conditional distributions is not straightforward. We propose a conditional kernel distributionally robust optimization (CKDRO) method that enables robust decision making under conditional distributions through kernel DRO and the conditional mean operator in the reproducing kernel Hilbert space (RKHS). In particular, we consider problems where there is a correlation between the unknown variable y and an auxiliary observable variable x. Given past data of the two variables and a queried auxiliary variable, CKDRO represents the conditional distribution P(y|x) as the conditional mean operator in the RKHS space and quantifies the ambiguity set in the RKHS as well, which depends on the size of the dataset as well as the query point. To justify the use of RKHS, we demonstrate that the ambiguity set defined in RKHS can be viewed as a ball under a metric that is similar to the Wasserstein metric. The DRO is then dualized and solved via a finite dimensional convex program. The proposed CKDRO approach is applied to a generation scheduling problem and shows that the result of CKDRO is superior to common benchmarks in terms of quality and robustness.

Chen, Yuxiao↗
DOE OSTI2026

Establishment of a Vertically Integrated Domestic Manufacturing Process for Production of Substrates Needed for Manufacture of Gas Diffusion Layers

In this project, AvCarb, LLC evaluated the baseline performance metrics of commercial carbon veils and their corresponding Gas Diffusion Layers (GDLs) with the goal of establishing an optimized, vertically integrated production system for wet-laid nonwoven substrates used in gas diffusion media for electrochemical energy storage and conversion devices. Mechanical testing and microstructural characterization were conducted and used to develop a multiscale computational model capable of simulating and predicting the performance of GDLs in fuel cells. Although the project successfully generated foundational transport and modeling data, it was terminated prior to identifying the critical GDL design parameters necessary for full optimization. The program aimed to improve carbon veil fabrication through enhanced fiber dispersion, fiber-fiber adhesion control, and improved web formation, enabling the production of high-quality, uniform substrates. Simulations were intended to guide mixing and solution delivery system design and process conditions, followed by production-scale trials to evaluate fiber dispersion, web uniformity, and mechanical robustness. At full deployment, the proposed production line would have been capable of producing approximately 650,000 m² of carbon veil annually. This capability remains strategically important, as the United States currently lacks a domestic source of wet-laid nonwoven carbon substrates that satisfy the stringent quality requirements for fuel-cell GDLs and electrolyzers representing an ongoing supply-chain vulnerability. Beyond supply-chain benefits, the project established a robust benchmarking dataset for existing commercial carbon veils while advancing next-generation material concepts targeting improved performance and manufacturing consistency.

Olson, Cynthia Lemay↗
NASA NTRS2016

Characterization of Evidence for Human System Risk Assessment

Understanding the kinds of evidence available and using the best evidence to answer a question is critical to evidenced-based decision-making, and it requires synthesis of evidence from a variety of sources. Categorization of human system risks in spaceflight, in particular, focuses on how well the integration and interpretation of all available evidence informs the risk statement that describes the relationship between spaceflight hazards and an outcome of interest. A mature understanding and categorization of these risks requires: 1) sufficient characterization of risk, 2) sufficient knowledge to determine an acceptable level of risk (i.e., a standard), 3) development of mitigations to meet the acceptable level of risk, and 4) identification of factors affecting generalizability of the evidence to different design reference missions. In the medical research community, evidence is often ranked by increasing confidence in findings gleaned from observational and experimental research (e.g., "levels of evidence"). However, an approach based solely on aspects of experimental design is problematic in assessing human system risks for spaceflight. For spaceflight, the unique challenges and opportunities include: (1) The independent variables in most evidence are the hazards of spaceflight, such as space radiation or low gravity, which cannot be entirely duplicated in terrestrial (Earth-based) analogs, (2) Evidence is drawn from multiple sources including medical and mission operations, Lifetime Surveillance of Astronaut Health (LSAH), spaceflight research (LSDA), and relevant environmental & terrestrial databases, (3) Risk metrics based primarily on LSAH data are typically derived from available prevalence or incidence data, which may limit rigorous interpretation, (4) The timeframe for obtaining adequate spaceflight sample size (n) is very long, given the small population, (5) Randomized controlled trials are unattainable in spaceflight, (6) Collection of personal and environmental data on the astronaut population may create opportunities for advanced analytics and human-environment modeling that goes beyond that achieved in isolated experimental designs; and (7) Translation of relevant research to operations is a complex, transdisciplinary enterprise in which the approach must apply across the physical, biological, behavioral, and social sciences. The approach to synthesizing evidence must address both source and fidelity of data, and reflect the most general attributes of quality of evidence in science and engineering: reliability and validity. The authors are developing a two-factor approach which includes the various kinds of evidence required to understand risks and for the integrated interpretation of all evidence that is essential to develop standards and countermeasures. A unified framework for aggregating and assessing different kinds of evidence provides a consistent, traceable, evidence-based decision-making process to translate research to operations in an environment where engineers, scientists, physicians, and managers all engage in analyzing the trade space of vehicle design, standards, requirements and solutions for spaceflight.

Steinberg, S. L.↗
DOE OSTI2020

Ambient Mass Spectrometry Rapid Analysis Methods for Trace Inorganics

The ability to rapidly detect and characterize the chemistry of trace quantities of inorganic species has multiple potential applications, including nuclear safeguards, nuclear forensics, and environmental monitoring. Some mass spectrometric methods currently exist, but they are costly and time-intensive, often destroying chemical information in the process of ionization. Ambient mass spectrometry is a relatively new analytical method in which samples are introduced to the mass spectrometer from atmospheric pressure, rather than from high vacuum. Ambient MS can operate with gentler ionization methods such as electrospray ionization (ESI) and paper-spray ionization (PSI), reducing fragmentation and preserving the chemical information of the analyte. This may be useful for determining the intended purpose of material in a sample. Strontium, part of nuclear fallout, is analyzed here to test these new analytical methods for viability. Electrospray ionization mass spectrometry functions by spraying the sample as a solution through a needle charged to a few kV. A nebulizing gas flows out along with the sample to create the spray before droplets enter a drying chamber with heated drying gas to remove the solvent molecules. From there bare analyte ions enter the mass spectrometer. This method has the advantage of relatively fast sample preparation and stable ionization. More stable ionization produces higher-quality data. Paper-spray ionization mass spectrometry operates by applying sample to a paper triangle. The paper triangle is then electrified in a metal clamp to a few kV before spray solvent is applied. The spray solvent transports analyte in the sample to the tip, where it is ionized and enters the mass spectrometer. One advantage of this method is reduced sample preparation - an analyst can even swipe a surface and cut a triangle directly from the swipe. The data analysis serves to rapidly process tens of samples all at once. The mass spectrometer outputs a file containing a few hundred spectra for each sample. The pipeline processes each one to identify peaks of interest. A full palette of data analysis metrics are generated, including plots of the raw data and plots of calculated values such as isotopic ratios. The pipeline filters the raw data to remove noise and background peaks, as well as peaks representing detector saturation. A heat map is generated to show the overall spectrum in mass-time- intensity space. Post-processing, we obtained the desired calibration curves, demonstrating that both ESI-MS and PSI-MS are viable methods for quantifying trace amounts of inorganics while retaining their chemical speciation information. ESI-MS demonstrated a linear response region of 100 ppb to 20 ppm, with a limit-of-detection of around 100 ppb. At top left is an example spectrum with the {sup 88}SrNO{sub 3} peak identified to show the retention of chemical speciation. PSI-MS showed a linear response region of around 1 ppb to 1 ppm, with a limit-of-detection of near 1 ppb. PSI-MS produced more erratic results than ESI-MS, likely due to the variability in ionization resulting from manual positioning of the paper triangle near the MS inlet. Future efforts will focus on improving the repeatability of PSI-MS as well as analyzing other species of interest such as cerium and uranium. Improvements will also be made to create more reliable data extraction and intelligent data filtering.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Adversarial sampling of unknown and high-dimensional conditional distributions

Many engineering problems require the prediction of realization-to-realization variability or a refined description of modeled quantities. In that case, it is necessary to sample elements from unknown high-dimensional spaces with possibly millions of degrees of freedom. While there exist methods able to sample elements from probability density functions (PDF) with known shapes, several approximations need to be made when the distribution is unknown. In this paper the sampling method, as well as the inference of the underlying distribution, are both handled with a data-driven method known as generative adversarial networks (GAN), which trains two competing neural networks to produce a network that can effectively generate samples from the training set distribution. In practice, it is often necessary to draw samples from conditional distributions. When the conditional variables are continuous, only one (if any) data point corresponding to a particular value of a conditioning variable may be available, which is not sufficient to estimate the conditional distribution. This work handles this problem using an a priori estimation of the conditional moments of a PDF. Herein, two approaches, stochastic estimation, and an external neural network are compared for computing these moments; however, any preferred method can be used. The algorithm is demonstrated in the case of the deconvolution of a filtered turbulent flow field. It is shown that all the versions of the proposed algorithm effectively sample the target conditional distribution with minimal impact on the quality of the samples compared to state-of-the-art methods. Additionally, the procedure can be used as a metric for the diversity of samples generated by a conditional GAN (cGAN) conditioned with continuous variables.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2015

SU-E-I-04: Improving CT Quality for Radiation Therapy of Patients with High Body Mass Index Using Iterative Reconstruction Algorithms

Purpose: Iterative reconstruction (IR) algorithms are developed to improve CT image quality (IQ) by reducing noise without diminishing spatial resolution or contrast. The CT IQ for patients with a high Body Mass Index (BMI) can suffer from increased noise due to photon starvation. The purpose of this study is to investigate and to quantify the IQ enhancement for high BMI patients through the application of IR algorithms. Methods: CT raw data collected for 6 radiotherapy (RT) patients with BMI, greater than or equal to 30 were retrospectively analyzed. All CT data were acquired using a CT scanner (Somaton Definition AS Open, Siemens) installed in a linac room (CT-on-rails) using standard imaging protocols. The CT data were reconstructed using the Sinogram Affirmed Iterative Reconstruction (SAFIRE) and Filtered Back Projection (FBP) methods. IQ metrics of the obtained CTs were compared and correlated with patient depth and BMI. The patient depth was defined as the largest distance from anterior to posterior along the bilateral symmetry axis. Results: IR techniques are demonstrated to preserve contrast and reduce noise in comparison to traditional FBP. Driven by the reduction in noise, the contrast to noise ratio is roughly doubled by adopting the highest SAFIRE strength. A significant correlation was observed between patient depth and IR noise reduction through Pearson’s correlation test (R = 0.9429/P = 0.0167). The mean patient depth was 30.4 cm and the average relative noise reduction for the strongest iterative reconstruction was 55%. Conclusion: The IR techniques produce a measureable enhancement to CT IQ by reducing the noise. Dramatic noise reduction is evident for the high BMI patients. The improved CT IQ enables more accurate delineation of tumors and organs at risk and more accuarte dose calculations for RT planning and delivery guidance. Supported by Siemens.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Long-Range Biometric Identification in Real World Scenarios: A Comprehensive Evaluation Framework Based on Missions

The considerable body of data available for evaluating biometric recognition systems in Research and Development (R&D) environments has contributed to the increasingly common problem of target performance mismatch. Biometric algorithms are frequently tested against data that may not reflect the real world applications they target. From a Testing and Evaluation (T&E) standpoint, this domain mismatch causes difficulty assessing when improvements in State-of-the-Art (SOTA) research actually translate to improved applied outcomes. This problem can be addressed with thoughtful preparation of data and experimental methods to reflect specific use-cases and scenarios.To that end, this paper evaluates research solutions for identifying individuals at ranges and altitudes, which could support various application areas such as counterterrorism, protection of critical infrastructure facilities, military force protection, and border security. We address challenges including image quality issues and reliance on face recognition as the sole biometric modality. By fusing face and body features, we propose developing robust biometric systems for effective long-range identification from both the ground and steep pitch angles. Preliminary results show promising progress in whole-body recognition. This paper presents these early findings and discusses potential future directions for advancing long-range biometric identification systems based on mission-driven metrics.

Aykac, Deniz↗
DOE OSTI2016

SU-F-T-601: Equivalence of CyberKnife and TrueBeam STx for Spine Irradiation

Purpose: To evaluate the equivalence of spine SBRT treatment plans created in Eclipse for the TrueBeam STx (Varian Medical System, Palo Alto, CA) compared to plans using CyberKnife and MultiPlan (Accuray, Sunnyvale, CA). Methods: CT data and contours for 23 spine SBRT patients previously treated using CyberKnife (CK) were exported from MultiPlan treatment planning system into Eclipse where they were planned using static IMRT 6MV coplanar beams. Plans were created according to the original prescription dose and fractionation schedule while limiting spinal dose according to the RTOG 0631 protocol and maintaining target coverage comparable to the original CK plans. Plans were evaluated using new conformity index (nCI), homogeneity index (HI), dose-volume histogram data, number of MU, and estimated treatment time. To ensure all Eclipse plans were deliverable, standard clinical IMRT QA was performed. The plan results were matched with their complimentary CK plans for paired statistical analysis. Results: Plans generated in Eclipse demonstrated statistically significant (p<0.01) improvements compared to complimentary CK plans in median values of maximum spinal cord dose (17.39 vs. 18.12 Gy), RTOG spinal cord constraint dose (14.50 vs. 16.93 Gy), nCI (1.28 vs. 1.54), HI (1.13 vs. 1.27), MU (3918 vs. 36416), and estimated treatment time (8 vs. 48 min). All Eclipse generated plans passed our clinically used protocols for IMRT QA. Conclusion: CK spine SBRT replanned utilizing Eclipse for LINAC delivery demonstrated dosimetric advantages. We propose improvements in plan quality metrics reviewed in this study may be attributed to dynamic MLCs that facilitate treatment of complicated geometries as well as posterior beams ideal for centrally located and/or posterior targets afforded by gantry-based RT delivery.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Methodological Guidelines on Net Energy Analysis of Photovoltaic Electricity

Net Energy Analysis (NEA) is a structured, comprehensive method of quantifying the extent to which a given energy source is able to provide a net energy gain (i.e., an energy surplus) to the end user, after accounting for all the energy losses occurring along the chain of processes that are required to exploit it (i.e., for its extraction, processing and transformation into a usable energy carrier, and delivery to the end user), as well as for all the additional energy 'investments' that are required in order to carry out the same chain of processes. However, this general framework leaves the individual practitioner with a range of choices that can affect the results and thus, the conclusions of a NEA study. The current IEA PVPS guidelines were developed to provide guidance on assuring consistency, balance, and quality to enhance the credibility and reliability of the results from photovoltaic (PV) NEAs. The guidelines represent a consensus among the authors -- PV NEA experts in North America, Europe, and Asia -- for assumptions made on PV performance, process inputs and outputs, methods of analysis, and reporting of the results. Guidance is given on photovoltaic-specific parameters used as inputs in NEA and on choices and assumptions in inventory data analysis and on implementation of modelling approaches. A consistent approach towards system modelling, the functional unit, the system boundaries and allocation aspects enhances the credibility of PV electricity NEA studies and enables balanced NEA-based comparisons of different electricity producing technologies. This document provides an in-depth discussion of a common metric of NEA, namely the energy return on investment (EROI), and how this is to be interpreted vis-a-vis the deceptively similar-sounding metrics in the field of Life Cycle Assessment (LCA): cumulative energy demand (CED) and non-renewable cumulative energy demand (nr-CED) per unit output. Specifically, a number of key differences are highlighted between these metrics as applied to electricity production systems. Transparency in reporting is of the utmost importance as parameters vary with geographical zones, and a system's boundary conditions and modelling approach can affect the findings significantly. This guideline lists 16 items that should be reported in every NEA study of PV electricity.

14 SOLAR ENERGY↗
DOE OSTI2019

Integrated Commercial Carbon Capture and Storage (CCS) Prefeasibility Study at Rock Springs Uplift, Wyoming (Final Report)

To establish the potential of safe, long-term commercial-scale carbon capture & storage (CCS) in southwestern Wyoming, the University of Wyoming formed a CCS coordination team (CCT) to complete a Phase I CarbonSAFE pre-feasibility study adjacent to the region’s largest coal-fired power plant, the Jim Bridger Plant (JBP). The study’s primary objective was to evaluate the potential of storing 50+ million metric tons of anthropogenic carbon dioxide (CO 2 ) in stacked reservoirs over a period of 25 years. In addition, this integrated study evaluated related aspects of carbon capture utilization & storage (CCUS) at the study site, including: (1) the CO 2 character of post-combustion flue gas from JBP and the technical capacity for retrofitting proven commercial-scale CO 2 capture technology relative to CO 2 source character and technological constraints; (2) utilization of the existing CO 2 pipeline network in the immediate vicinity of JBP for enhanced oil recovery (CO 2 -EOR) and utilizing that network as part of a regional hub for CO 2 storage from other anthropogenic sources; (3) assessment of the challenges and benefits of meeting all of CarbonSAFE programmatic Phase I goals relative to Wyoming’s carbon management regulatory framework; and (4) assessment of the community and environmental factors that may impact CCS at JBP. With respect to storage, we analyzed the potential of stacked Mesozoic reservoirs at the study site, focusing on the Entrada and Nugget sandstones. The reservoir response to injection simulations varied greatly between formations, though they both were capable of CO 2 storage. The Entrada Sandstone is relative thin (~40’) and heterogeneous; storage using one injection and production well was limited to 1.2 Mt over 25 years at the study site. The Nugget Sandstone is much thicker (> 400’) and generally homogeneous; storage using one injection and production well approached 15.0 Mt over 25 years; this scenario suggests that the storage potential in the Nugget Sandstone is close to the estimated P90 calculation of 9.6 Mt/mi 2 as models indicate a storage potential of 8.3 Mt/mi 2 at saturation. Both reservoirs’ storage capacity benefited greatly from coupling an injection well with an offset production well; total storage capacity increased and reservoir pressure stayed below critical thresholds. This indicates that implementing a pressure management strategy at the study site would optimize storage and decrease risk. Seals associated with the targeted reservoirs were shown to be capable of retaining the potential storage volumes, and the site benefits from multiple seal redundancies (i.e. > 7,000’ of seal formations). Furthermore, assessment of different reservoir variables (e.g., fluid composition, pressure, etc.) indicates all reservoirs at the study site are isolated. Storage assessments from Mesozoic reservoirs were coupled with simulations from the site’s Paleozoic reservoirs (the Weber Sandstone and Madison Limestone), which were the focus of previous study. The fully integrated stacked reservoir pre-feasibility studies suggest that the Nugget and Madison formations alone could safely store 50 million metric tons of CO 2 within the study site. Furthermore, both formations are true saline reservoirs near JBP, making them ideal target reservoirs for CO 2 sequestration. Analysis of Wyoming’s laws and regulations confirm that the State has a favorable policy environment for CCS/CCUS. The project’s economic models estimated that the following revenues collectively are sufficient to finance the project over its lifespan if CO 2 utilization is coupled with CCS via the following strategies: (1) sales of CO 2 for EOR (approximately $69 million/year); (2) use of CO 2 tax credits such as amended §45Q and §48A (approximately $484 million total); and (3) sales of low-carbon electricity and marketable carbon offset/credits into carbon-constrained markets (approximately $11-$17 million/year) to West Coast consumers. These revenue estimates are broadly consistent with other studies that concluded arbitrage in JBP retail electricity sales between California and Wyoming could help to support the cost of deploying CCS at JBP. The models also indicate that maintaining a successful business case for long-term CCS -- wherein associated capture, transportation and operational costs are fully addressed at JBP -- is reliant on numerous external factors that complicate predictions. This study suggests that this site meets the CarbonSAFE program’s requirements of being able to feasibly store 50+ million metric tons of CO 2 over 25 years within the site’s stacked reservoirs, especially if coupled with pressure management strategies. This study suggests that the Nugget Sandstone and the Madison Limestone are two of the best reservoirs in the State with respect to overall storage capacity. In addition, the site’s proximity to existing CO 2 transportation networks, CO 2 -EOR opportunities, ability to sell to markets that value low-carbon electricity, Wyoming’s existing carbon regulatory framework, and a public that is well-educated with respect to energy markets and issues collectively increase the potential for implementing commercial-scale CCS adjacent to the JBP. With respect to the available technical and non-technical site data, and the reservoir quality at the study site, we suggest that this site is capable of moving directly into Phase III of the CarbonSAFE program.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2015

SU-E-T-341: DVH-Based Comparison of True 3D Measurements to a Delta4 System

Purpose: Delta4 dosimetric software can be used to calculate DVH-based metrics for patient-specific quality assurance from measurements made by a Delta4 QA device. This study investigates the effectiveness of a novel transform method that transposes measurements made with a full-density 3D dosimeter onto patient anatomy, enabling the calculation of DVHs. This allows for DVH comparisons from the transformed dose distribution, which are based on true 3D measurements, to those from the Delta4 system, which are based on semi-3D measurements and interpolation. Methods: A double-arc VMAT treatment for a head-and-neck case was delivered to a 1kg PRESAGE 3D dosimeter inserted into a polyurethane head phantom. The dosimeter was readout using an in-house optical-CT scanner to gather full-density 3D dosimetric data. The transform method is achieved by multiplication of the measured doses with a “transformation matrix” which accounts for heterogeneities and differences in geometry between the patient and the phantom. The transformation matrix is a voxel-by-voxel division of the patient planned dose by the phantom planned dose, both calculated in the treatment planning system (Eclipse). The transformed distribution was then overlaid on the patient CT data, enabling the calculation of DVHs. The same VMAT treatment was delivered to the Delta4 phantom and DVH data was calculated using its associated software. Results: The transformed dose distribution showed good agreement with calculated patient values, determined by similarity in dose profiles between the two distributions and a 3D gamma index passing rate of 94.87% for 3%/3mm criteria. For every structure contained within the dosimeter volume, the transformed DVHs demonstrated better agreement than the Delta4 DVHs, when compared to the values calculated in the treatment planning system. Conclusion: The coupled technique of full-density 3D dose measurements and the presented transform method enables clinical patient-specific quality assurance data that is more accurate than the semi-3D Delta4 system. This work was supported by NIH R01CA100835.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

AI Model Benchmarking for Nonproliferation Applications: Steel Thread Benchmarking Task Force Technical Report (Rev. 2)

Steel Thread is a NA-22 venture that seeks to build trustworthy, reliable AI models that can be used in a wide variety of nonproliferation tasks. A key aspect of building these models is developing appropriate benchmarks and evaluation methods, which will enable the venture to identify and adapt models to provide the most value in the nonproliferation domain. Benchmarks must be relevant to key tasks in this domain, such as question answering, information retrieval, document summarization and classification, consensus analysis, and image and data analysis. This report 1) provides an overview of benchmark design, evaluation, and challenges; 2) reviews a variety of open benchmarks, with a focus on language models and tasks; and 3) identifies benchmarks that are most relevant to Steel Thread. This report is intended to serve as a basis for further efforts to classify and evaluate benchmarks and their correlation with success on nonproliferation-specific tasks. The Steel Thread venture has defined benchmarks to be a particular combination of a dataset (or datasets) and a metric (or metrics) conceptualized as representing one or more specific tasks or sets of abilities for a specific modality. It is adopted by a research community as a shared framework for comparing methods.1 It includes 1) Data: Labeled (a designated subset not used for training, which could be all the data), 2) Metric: A way to quantify performance, 3) Task/Ability: What the benchmark is testing, 4) Protocol: A structured and repeatable evaluation process, 5) Baseline/Reference Model: For comparison; could be statistical, rule-based, SME-derived, or another model, and 6) Maintenance Plan: to update with new information over time; important for long-term utility. For further clarity, the definition includes what a benchmark, in this context, is not. It is not a corpus of training data, specific to a model (it is intended to apply to a range of models), a universal evaluation of performance, a guarantee that the ‘top’ model on the leaderboard will be the best fit for every specific use case, an all-encompassing proof of a model’s universal quality, nor is it a one-size-fits-all measure of success. It does not cover every real-world constraint (like operational, ethical, or cost considerations), a systems integration test, or a unit test. This definition was inspired by and resulted from discussions within the Steel Thread Benchmarking Task Force. This group was formed to define what we would mean as a benchmark within Steel Thread but persisted as the need to develop a thorough understanding of the large and expanding existing benchmarking space. This technical report is a result of the group’s divide and conquer approach to exploring this space. The release of benchmarks might not be progressing as quickly as model development, but it is moving very fast, as many benchmarks quickly become saturated, when state-of-the-art models score so close to the benchmark’s ceiling that their results are virtually indistinguishable. At that point, the test no longer differentiates between new systems, so researchers usually stop reporting scores as the benchmark no longer informs about improvements from the next generation of models. In the OpenAI announcement of GPT-5, they reported results on six flagship public benchmarks (AIME 2025, SWE-bench Verified, Aider Polyglot, MMMU, HealthBench Hard, GPQA) but the full system-card covers roughly thirty-five separate evaluations, comprising hundreds of test task items in total. There have been some efforts to summarize benchmarks in specific fields, like for text-to-image generation, but these surveys have had a narrow methodology scope. Therefore, a comprehensive survey of all benchmarks or even all benchmarks that could be relevant to Steel Thread is outside of the scope of this report. We chose some specific benchmarks to investigate in detail.

97 MATHEMATICS AND COMPUTING↗