Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Information Sharing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Sharing the Sun Community Solar Project Data

This database represents a list of community solar projects, complete and pending, identified through various sources. The dataset is updated multiple times per year. The current version is the first file located below. Previous versions of the dataset published before June of 2024 can be found in the dataset below labeled “ARCHIVE_Sharing the Sun Community Solar Project Data_Before 06.24.“ The list has been reviewed but errors may exist, and the list may not be comprehensive. Errors in the sources e.g. press releases may be duplicated in the list. Blank spaces represent missing information. NLR invites input to improve the database including, to correct erroneous information, add missing projects, fill in missing information, and remove inactive projects. Updated information can be submitted to Sudha Kannan ( sudha.kannan@nlr.gov ).

14 SOLAR ENERGY↗
DOE OSTI2021

Machine Intelligence to Detect, Characterise, and Defend against Influence Operations in the Information Environment

Social media has enabled a new era of manipulation in the information and cognitive domains. Deceptive content—misleading, falsified, and fabricated—is routinely created and spread in the modern social media environment with the intent to create confusion and widen political and social divides, and exploit the societal conflict exacerbated by these divides in the real-world (aka physical domain). Such disinformation campaigns demonstrate a threat to the integrity of economic, political, cultural, public health, and national security institutions around the world. In this work we overview our artificial intelligence (AI) capabilities to detect, describe, and defend against information operations on Twitter as an example social platform to understand the influence of misleading and falsified content diffusion and better enable those charged with defending against such manipulation to enable responsive parties to counter it. We first present novel linguistically-informed deep learning (DL) models for misinformation and disinformation detection, and present an in-depth linguistic analysis of psycho-linguistic markers across broad deception categories. We then demonstrate how our models perform in the multilingual and multimodal setting and categorize falsified and misleading content based on the intent to deceive. We also provide a large-scale analysis to describe user behavior and spread patterns while engaging with deceptive content and report novel findings about the immediate diffusion of deceptive content by characterizing the vulnerable sub-populations and their demographics, and explicitly measuring speed and scale of deception spread to uncover who shares deceptive content, how quickly, how much, and how evenly. In addition, we measure audience reactions to misinformation and disinformation at scale, distinguishing the reactions of users identified as bots versus humans. Finally, we take advantage of deep translation and generation models to create unique solutions for real-time defense against digital deception and discuss how to apply causal inference to prescribe and intervene into strategic communications jointly across information, cognitive, and physical domains.

artificial intelligence, deep learning, neural lan↗
DOE OSTI2024

Data-Informed Synthetic Networks of Water Distribution Systems for Resilience Analysis in Puerto Rico

The increasing potential of infrastructure disruptions calls for high-quality infrastructure models to be used in resilience analysis and decision making. Unfortunately, many utilities and communities do not have access to accurate and detailed models due to a lack of data and resources. Furthermore, security restrictions on sharing infrastructure models present roadblocks to research, analysis, and decision making. Recent advances in the development of synthetic water distribution models provide a potential solution to this problem. There is an opportunity to improve these methods by leveraging incomplete pipe datasets to aid synthetic network generation. To address this gap, we developed a methodology for synthetic network generation that incorporates partial pipe data using a modification of the minimum cost flow algorithm for network generation and pipe sizing. This methodology demonstrates how partial pipe data can be leveraged to improve site-specific synthetic network generation. For the study area of Mayagüez, Puerto Rico, a synthetic model generated using 50% of real pipe data matches the pressure of the validation system with an average error of 23.5 m of head, which improves upon the average error of 31.6 m of head produced by a synthetic model generated using no data of the real pipes. Additionally, synthetic networks are shown to replicate the pressure response under a disruption scenario of the validation network, suggesting potential use in resilience analysis.

resilience analysis↗
DOE OSTI2025

Machine learning guided selection of broad-spectrum epitope-specific functional antibodies for "Disease X"

Our project established and demonstrated a transfer learning framework that enables prediction of antibody–antigen interactions across related viruses. The approach focused on three major activities: 1. Conserved region and epitope identification – We compared viral protein structures and sequences to identify shared receptor-binding domains and neutralizing epitope regions across variants and related viruses. These conserved features formed the foundation for discovering broadly functional antibodies. 2. Machine learning model development – We built neural network–based models that integrate epitope features with antibody sequence information. Instead of relying solely on structural or physical properties, the models learned transferable patterns that describe antibody binding potential across different viral families. 3. Transfer learning and validation – Using SARS-CoV-2 and Ebola as source systems, we successfully transferred learned epitope features to predict antibody interactions for SARS CoV-1 and Marburg virus. Iterative cycles of dataset generation, retraining, and evaluation improved generalization and predictive power, ensuring the framework can adapt to new threats.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

CLM5 Simulations of Soil Moisture and Grain Carbon for CONUS at 0.125 degrees

This dataset provides 0.125° gridded simulations of soil moisture and crop grain carbon for the Contiguous United States (CONUS), generated using the Community Land Model version 5 (CLM5) with the biogeochemistry module enabled. The data covers a historical baseline (1980–2015) and mid-century future projections (2020–2055). Future projections are organized into two sets of scenarios to distinguish the impacts of different drivers: (1) Atmospheric Only (ATM-only): These scenarios apply future atmospheric forcings while holding land use and land cover change (LULCC) at historical baseline levels. The atmospheric forcings represent moderately versus severely hotter/drier atmospheric conditions (dynamically downscaled perturbed thermodynamics simulations based on CMIP6 SSP245 and SSP585 warming signals), each with cooler versus hotter Earth System Model temperature sensitivity instantiations. These scenarios are identified in the file names as atm45cooler, atm45hotter, atm85cooler, and atm85hotter. (2) Coupled Atmospheric and Land-Use (LAND+ATM): These scenarios apply future atmospheric forcing together with future LULCC by pairing atmospheric pathways with lower versus higher population/economic growth scenarios representing Shared Socioeconomic Pathways 3 and 5 (SSP3 and SSP5). These scenarios are identified in the file names as atm45cooler_ssp3, atm45hotter_ssp3, atm45cooler_ssp5, atm45hotter_ssp5, atm85cooler_ssp3, atm85hotter_ssp3, atm85cooler_ssp5, and atm85hotter_ssp5. Please refer to the README file for detailed information on file structure, variables, units, and data formats.

Yao, Lili [Pacific Northwest National Laboratory] ↗
DOE OSTI2026

CLM5 Simulations of Soil Moisture and Gross Primary Productivity for CONUS at 0.125 degrees

This dataset provides 0.125-degree gridded simulations of soil moisture and gross primary productivity (GPP) for the Contiguous United States (CONUS), generated using the Community Land Model version 5 (CLM5) with the biogeochemistry module enabled. The data covers a historical baseline (1980-2015) and mid-century future projections (2020-2055). Future projections are organized into two sets of scenarios to distinguish the impacts of different drivers: (1) Atmospheric Only (ATM): These scenarios apply future atmospheric forcings while holding land use and land cover (LULC) at historical baseline levels. The atmospheric forcings represent moderately versus severely hotter/drier atmospheric conditions (dynamically downscaled perturbed thermodynamics simulations based on CMIP6 SSP245 and SSP585 warming signals), each with cooler versus hotter Earth System Model temperature sensitivity instantiations. These scenarios are identified in the folder names as rcp45_cooler_near, rcp45_hotter_near, rcp85_cooler_near, and rcp85_hotter_near. (2) Coupled Atmospheric and Land-Use (LAND+ATM): These scenarios apply future atmospheric forcing together with future LULC by pairing atmospheric pathways with lower versus higher population/economic growth scenarios representing Shared Socioeconomic Pathways 3 and 5 (SSP3 and SSP5). These scenarios are identified in the file names as ssp3_rcp45_cooler_near, ssp3_rcp45_hotter_near, ssp5_rcp85_cooler_near, and ssp5_rcp85_hotter_near. Please refer to the "README_first.md" file for detailed information on file structure, variables, units, and data formats.

drought↗
DOE OSTI2025

Securing the Modern Grid: Federal Investments, Digitization, and Supply Chain Strategy

Across the United States (U.S.) grid expansion and modernization is underway, paving the way for accelerated load growth and intelligent resource management. Digitization of the grid is supported by several state and federal programs, providing support for utilities installing advanced metering infrastructure (AMI), AI-powered analytics systems, battery energy storage systems (BESS), and distributed energy resource management systems (DERMS) to transform the grid from a one-way power delivery system into an intelligent, responsive network that will enable faster load growth and power expansion of data centers for advanced artificial intelligence (AI) applications. The digital transformation of America's grid presents opportunity for increased efficiency and resiliency but also introduces new digital risks that require careful management. Digital equipment often contains several vulnerabilities such as unencrypted communication protocols, and persistent remote access capabilities that could be exploited to manipulate device settings, coordinate service disruptions, or inject false data into grid operations. These digital risks become particularly important as the grid must rapidly scale to support AI-driven data centers, which the administration has identified as essential for maintaining U.S. technological leadership and economic competitiveness. These vulnerabilities are compounded by supply chain realities: Chinese manufacturers currently produce 70-90% of essential grid components including inverters, batteries, and control systems, with the U.S. lacking domestic manufacturing capacity for critical assets like extra-high voltage transformers. Recent federal legislation has established Foreign Entity of Concern (FEOC) restrictions to address these risks, requiring projects to achieve escalating thresholds of non-FEOC content to receive tax credits while utilities work to expand sourcing channels for their supply chains and strengthen security measures. These restrictions arrive precisely when utilities face unprecedented electricity demand growth driven by the rapid growth in data centers, creating a considerable challenge: rapidly expanding infrastructure while navigating complex compliance requirements while lacking viable alternatives for many critical components. Idaho National Laboratory (INL) and its partners have developed practical approaches to help utilities navigate these intersecting challenges as they leverage federal investment to strengthen and grow the grid. These solutions include Cyber-Informed Engineering (CIE) principles that build resilience directly into systems, the Cirrus tool for secure cloud migration, and enhanced procurement guidance that embeds security requirements throughout equipment lifecycles. Federal initiatives, such as the Technical Assistance for Digital Assurance (TADA) project, provide direct support to utilities implementing these approaches while facilitating knowledge sharing across the industry. While these tools and frameworks cannot eliminate all risks inherent in foreign supply chain dependencies, they offer pragmatic pathways for strengthening security posture without sacrificing the deployment momentum essential to meeting surging electricity demand. Ultimately, securing America's digital energy infrastructure demands dedicated coordination across multiple fronts: building domestic supply chains, implementing robust digital assurance practices, and maintaining the aggressive modernization timeline necessary for reliability, resilience, and energy independence.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

HarDWR - Raw Water Rights Records

A dataset within the Harmonized Database of Western U.S. Water Rights (HarDWR). For a detailed description of the database, please see the meta-record v2.0. Changelog v2.0 - Switched source data from collecting records from each state independently to using the WestDAAT dataset v1.0 - Initial public release Description In order to hold a water right in the western United States, an entity, (e.g., an individual, corporation, municipality, sovereign government, or non-profit) must register a physical document with the state's water regulatory agency. State water agencies each maintain their own database containing all registered water right documents within the state, along with relevant metadata such as the point of diversion and place of use of the water. All western U.S. states have digitized their individual water rights databases, as well as geospatial data defining the areas in which water rights are managed. Each state maintains and provides their own water rights data in accordance with individual state regulations and standards. In addition, while all states make their water rights publicly available, each provides their records in unique formats, meaning that file types, field availability, and terms vary from state to state. This leads to additional challenges to managing resources which cross state lines, or conducting consistent multi-state water analyses. For the first version of HarDWR, we collected the water rights databases from 11 Western States of the United States. In order to preform regional analyses with the collected data, the raw records had to be harmonized into one single format. The Water Data Exchange (WaDE) is a program dedicated to the sharing of water-related data for the Western U.S. in a singular consistent format. Created by the Western States Water Council (WSWC) to facilitate the collection and dissemination of water data among WSWC's member states and the public, WaDE provides an important service for those interested in water resource planning and management in their focus region. Of the services which WaDE provides, the one of the most interesting is the WestDAAT dataset, which is a collection of water rights data provided by the 18 WSWC member states that have been standardized into a single format, much like we had done on a more limited scale with HarDWR v1. For this version of HarDWR we decided to use WestDAAT, specifically a snapshot created in Feburary 2024, as our water rights source data. A full explanation of the benefits gained from this switch can be found in the description of the updated Harmonized Water Rights Records v2.0, but in short it has allowed us to focus more of our efforts on answering research questions and gaining a more realistic understanding of how water rights are allocated. For more information on how the data for WestDAAT was collected, please see the WaDE data summary. Terms of Use While WaDE works directly with the state agencies to collect and standardize the water rights records, the ultimate authority for the water rights data remains the individual states. Each state, and their respective water right authorities, have made their water right records available for non-commercial reference uses. In addition, the states make no guarantees as to the completeness, accuracy, or timeliness of their respective databases, let alone the modifications which we, the authors of this paper, have made to the collected records. None of the states should be held liable for using this data outside of its intended use. As several of the states update their water rights databases daily, the information provided here is not the latest possible, and should not be used for legal purposes. WestDAAT itself has irregular updates. Additional questions about the data the source states provided should be directed to the respective state agencies (see methods.csv and organization.csv files described below). In addition, although data was presented here was not collected directly from the states, several states requested specifically worked disclaimers when sharing their data. These disclaimers are included here as an acknowledgement from where the water rights data is primarily sourced. Colorado: "The data made available here has been modified for use from its original source, which is the State of Colorado. THE STATE OF COLORADO MAKES NO REPRESENTATIONS OR WARRANTY AS TO THE COMPLETENESS, ACCURACY, TIMELINESS, OR CONTENT OF ANY DATA MADE AVAILABLE THROUGH THIS SITE. THE STATE OF COLORADO EXPRESSLY DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. The data is subject to change as modifications and updates are complete. It is understood that the information contained in the Web feed is being used at one's own risk." Montana: "The Montana State Library provides this product/service for informational purposes only. The Library did not produce it for, nor is it suitable for legal, engineering, or surveying purposes. Consumers of this information should review or consult the primary data and information sources to ascertain the viability of the information for their purposes. The Library provides these data in good faith but does not represent or warrant its accuracy, adequacy, or completeness. In no event shall the Library be liable for any incorrect results or analysis; any direct, indirect, special, or consequential damages to any party; or any lost profits arising out of or in connection with the use or the inability to use the data or the services provided. The Library makes these data and services available as a convenience to the public, and for no other purpose. The Library reserves the right to change or revise published data and/or services at any time." Oregon: "This product is for informational purposes and may not have been prepared for, or be suitable for legal, engineering, or surveying purposes. Users of this information should review or consult the primary data and information sources to ascertain the usability of the information." File Descriptions The unmodified February, 2024 WestDAAT snapshot is composed of nine files. Below is a brief description of each file, as well as how they were utilized for HarDWR. WaDEDataDictionaryTerms.xlsx: As the file's name implies, this is a data dictionary for all of the below named files. This file describes the column names for each of the following files, with the exception of citation.txt which does not have any columns. The descriptions for each file are divided by tab,with the same name as their associated file, within this document. allocationamount.csv: The "main" file of the group, it contains the water right records for each state. Of particular note, each water right is broken down into one or more water allocations. Allocations may be withdrawn from one or more locations, or even multiple allocations associated with a particular location. This is a more subtle and realistic representation of how water is used than what was available in the first version of HarDWR. For the records from some states, this can mean that multiple allocations listed under a single right will appear as rows within this file. citation.txt: A combination of contact information for WaDE personnel, disclaimer about how the data should be used, and guidelines for citing WestDAAT. methods.csv: A file describing the source and method by which WaDE collected water rights data from each state. organization.csv: A file listing the water rights authoritative agencies for each state. sites.csv: This file provides the geographic, and other descriptors, of the physical location of allocations, called 'sites'. To reiterate, it is possible for one allocation to be associated with multiple sites, as well as one site to be associated with multiple allocations. The two descriptors which we were most interested in where the site's coordinates, as well as whether the site was classified as a Point of Diversion (POD) or a Place of Use (POU). As a general rule, PODs are geographic points, while POUs are areas typically represented as property boundaries or irregularly shaped polygons. sites_pouGeometry.csv: For those allocations with a POU site, this file contains the defining points for the associated polygons. variables.csv: A file describing the units in which an allocation's water amount is reported within WestDAAT. This information is essentially a repeat of the 'AllocationFlow_CFS' and 'AllocationVolume_AF' columns within allocationamount.csv, at least for our purposes. watersources: This file describes the source of water from which each site extracts from. For our purposes, this table was used to determine whether the water came from Surface Water, Groundwater, or Unspecified Water.

Lisk, Matthew↗
DOE OSTI2023

Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations

Physics-informed neural networks have emerged as an alternative method for solving partial differential equations. However, for complex problems, the training of such networks can still require high-fidelity data which can be expensive to generate. To reduce or even eliminate the dependency on high-fidelity data, we propose a novel multi-fidelity architecture which is based on a feature space shared by the low- and high-fidelity solutions. In the feature space, the projections of the low-fidelity and high-fidelity solutions are adjacent by constraining their relative distance. The feature space is represented with an encoder and its mapping to the original solution space is effected through a decoder. In conclusion, the proposed multi-fidelity approach is validated on forward and inverse problems for steady and unsteady problems described by partial differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Bioenergy Research Centers Data Sharing Portal

The bioenergy.org website is the end product of the Data Sharing Shared Research objective for the Bioenergy Centers. The objective is to Enhance BRC data legacy through the development and use of shared software tools to make previously published datasets more findable and accessible through the Inter-BRC Data Products Portal, and to explore the use of generative AI to assist in the exploration of published datasets. The software repository is released per the license information below.

Thrower, Nicholas↗
DOE OSTI2025

Status Report on Design of In-situ Thermomechanical Testing at LANSCE

Nuclear fuel encounters severe thermomechanical environments in which its mechanical response is determined by its microstructure, temperature and stress level histories. Simulating the response of such microstructures is crucial for predicting both performance and transient fuel mechanical responses and experimental verification of such predictions is therefore of great interest. While most of the deformation in a nuclear fuel rod occurs in the cladding, deformation of the fuel itself is still of interest with deformation mechanisms at operating temperature and above including creep, swelling, cracking as well as pellet-clad interaction. Characterization of these properties and understanding of the underlying deformation phenomena at operating or excursion temperatures is therefore of great importance for development and ultimately licensing of improved and novel nuclear fuel forms. Diffraction techniques offer unique insight on the atomistic (e.g. crystal structure) and microstructure (e.g. phase transformations, texture, defects) length scales and have a long history of providing unique data to inform relevant deformation models that enable the required predictive capabilities. For example, dislocations lead to diffraction peak broadening that can be characterized to estimate the dislocation density and study the role of dislocations on the deformation while measuring lattice strains allows to studie load sharing in two phase materials. In this report the requirements for a sample environment for high temperature deformation of nuclear fuels are defined. The HIPPO neutron time-of-flight diffractometer at LANSCE will host this sample environment and is also described. This instrument covers diffraction angles from 140° to 40° and is also equipped with an event-mode neutron imaging detector system, enabling energy-resolved neutron imaging in parallel with the diffraction that could measure sample temperature from Doppler broadening of neutron absorption resonances or measure pore densities from changes in the attenuation. Designs of devices to characterize thermomechanical properties of nuclear fuel without diffraction are also considered to guide the design. While this report is focused on applications for nuclear fuels, the device can also characterize cladding, moderator or structural materials and therefore contribute to other fields of research and development for advanced reactors. The temperatures planned to be reached are above 2000℃, thus enabling characterization of LWR reactor fuels under accident scenarios but also reaching temperatures of fuels developed for nuclear thermal propulsion and providing opportunities to characterize those. In conjunction with the energy-resolved neutron imaging detector, this setup would allow to measure neutron cross-sections at high temperatures, filling a gap towards development of reactors operating at high temperatures.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

A High-resolution Regional Wave Resource Characterization For The U.S. West Coast

Objectives/Scope: Wave resource characterization is a critical step for wave energy converter deployment in the coastal ocean and relies on long-term, high-resolution wave datasets. This study presents a detailed modeling study of the wave resource along the U.S. West Coast (Washington, Oregon, and California), a coastal region that was identified with high wave energy potential in earlier studies. Methods, Procedures, Process: The wave hindcast covers a 32-year period from 1979 to 2010 and is based on a multi-resolution, unstructured-grid SWAN model framework. Model configuration closely follows and meets the requirements recommended by the International Electrotechnical Commission Technical Specification (IEC TS) for wave energy resource assessment and characterization (Class 2 - feasibility study). The model domain covers the entire U.S. Exclusive Economic Zone (EEZ) in the West Coast and has a spatial resolution varying from ~300 m in the nearshore region (20 km from the shoreline) to ~2500 m within the EEZ and ~5000 m at the open boundary, which extends beyond the EEZ. The model was forced by hourly 2-D wave spectra produced by a two-way nested WaveWatch III model, which covers the global ocean domain and the broader U.S. West Coast region domain with spatial resolutions of 0.5 degree and 10 arc-minutes, respectively. Both wave models are forced by hourly, 0.5-degree wind forcing obtained from NCEP’s Climate Forecast System Reanalysis (CFSR) product. Results, Observations, Conclusions: The standard model output for the SWAN model includes 3-hourly output for the six IEC wave resource parameters (e.g., omnidirectional wave power) at each grid point and hourly 2-D spectra at more than 50 NDBC buoys. Extensive model validation was achieved by comparing the six model-predicted IEC parameters with those derived from field observations at representative NDBC buoys. The error statistics indicated the model’s satisfactory performance. Further analyses were conducted to systematically evaluate the temporal and spatial distributions of wave energy potential and wave climate along the U.S. West Coast. Results suggest that Washington and Oregon coasts have similar nearshore wave resource, which is significantly higher than resources in Southern California. Strong seasonal variations are also observed, e.g., high wave energy tends to occur in the winter months. In summary, this study produced the first high-resolution, comprehensive dataset on wave energy distribution along the U.S. West Coast. Novel/Additive Information: The results are being used by the National Renewable Energy Laboratory to update the MHK Atlas, which was originally derived from NOAA’s 4-arc-minute WaveWatch III model output. In addition, the monthly averaged wave energy climatology dataset can be readily shared to support a variety of research and application efforts within the EEZ of the U.S. West Coast.

Wang, Taiping↗
DOE OSTI2022

PythonFOAM: In-situ data analyses with OpenFOAM and Python

Here, we outline the development of a general-purpose Python-based data analysis tool for OpenFOAM. Our implementation relies on the construction of OpenFOAM applications that have bindings to data analysis libraries in Python. Double precision data in OpenFOAM is cast to a NumPy array using the NumPy C-API and Python modules may then be used for arbitrary data analysis and manipulation on flow-field information. We highlight how the proposed wrapper may be used for an in-situ online singular value decomposition (SVD) implemented in Python and accessed from the OpenFOAM solver PimpleFOAM. Here, 'in-situ' refers to a programming paradigm that allows for a concurrent computation of the data analysis on the same computational resources utilized for the partial differential equation solver. In addition, to demonstrate parallel deployments, we deploy a distributed SVD, which collects snapshot data across the ranks of a distributed simulation to compute the global left singular vectors. Crucially, both OpenFOAM and Python share the same message passing interface (MPI) communicator for this deployment which allows Python objects and functions to exchange NumPy arrays across ranks. Subsequently, we provide scaling assessments of this distributed SVD on multiple nodes of Intel Broadwell and KNL architectures for canonical test cases such as the large eddy simulations of a backward facing step and a channel flow at friction Reynolds number of 395. Finally, we demonstrate the deployment of a deep neural network for compressing the flow-field information using an autoencoder to demonstrate an ability to use state-of-the-art machine learning tools in the Python ecosystem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

AI Model Benchmarking for Nonproliferation Applications: Steel Thread Benchmarking Task Force Technical Report (Rev. 2)

Steel Thread is a NA-22 venture that seeks to build trustworthy, reliable AI models that can be used in a wide variety of nonproliferation tasks. A key aspect of building these models is developing appropriate benchmarks and evaluation methods, which will enable the venture to identify and adapt models to provide the most value in the nonproliferation domain. Benchmarks must be relevant to key tasks in this domain, such as question answering, information retrieval, document summarization and classification, consensus analysis, and image and data analysis. This report 1) provides an overview of benchmark design, evaluation, and challenges; 2) reviews a variety of open benchmarks, with a focus on language models and tasks; and 3) identifies benchmarks that are most relevant to Steel Thread. This report is intended to serve as a basis for further efforts to classify and evaluate benchmarks and their correlation with success on nonproliferation-specific tasks. The Steel Thread venture has defined benchmarks to be a particular combination of a dataset (or datasets) and a metric (or metrics) conceptualized as representing one or more specific tasks or sets of abilities for a specific modality. It is adopted by a research community as a shared framework for comparing methods.1 It includes 1) Data: Labeled (a designated subset not used for training, which could be all the data), 2) Metric: A way to quantify performance, 3) Task/Ability: What the benchmark is testing, 4) Protocol: A structured and repeatable evaluation process, 5) Baseline/Reference Model: For comparison; could be statistical, rule-based, SME-derived, or another model, and 6) Maintenance Plan: to update with new information over time; important for long-term utility. For further clarity, the definition includes what a benchmark, in this context, is not. It is not a corpus of training data, specific to a model (it is intended to apply to a range of models), a universal evaluation of performance, a guarantee that the ‘top’ model on the leaderboard will be the best fit for every specific use case, an all-encompassing proof of a model’s universal quality, nor is it a one-size-fits-all measure of success. It does not cover every real-world constraint (like operational, ethical, or cost considerations), a systems integration test, or a unit test. This definition was inspired by and resulted from discussions within the Steel Thread Benchmarking Task Force. This group was formed to define what we would mean as a benchmark within Steel Thread but persisted as the need to develop a thorough understanding of the large and expanding existing benchmarking space. This technical report is a result of the group’s divide and conquer approach to exploring this space. The release of benchmarks might not be progressing as quickly as model development, but it is moving very fast, as many benchmarks quickly become saturated, when state-of-the-art models score so close to the benchmark’s ceiling that their results are virtually indistinguishable. At that point, the test no longer differentiates between new systems, so researchers usually stop reporting scores as the benchmark no longer informs about improvements from the next generation of models. In the OpenAI announcement of GPT-5, they reported results on six flagship public benchmarks (AIME 2025, SWE-bench Verified, Aider Polyglot, MMMU, HealthBench Hard, GPQA) but the full system-card covers roughly thirty-five separate evaluations, comprising hundreds of test task items in total. There have been some efforts to summarize benchmarks in specific fields, like for text-to-image generation, but these surveys have had a narrow methodology scope. Therefore, a comprehensive survey of all benchmarks or even all benchmarks that could be relevant to Steel Thread is outside of the scope of this report. We chose some specific benchmarks to investigate in detail.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A Framework for Multisector Scenarios of Outcomes for Well-Being and Resilience

Shared community scenarios of societal and environmental system changes have underpinned a broad range of research and assessment studies over the past several decades. These scenarios have largely aimed to address specific questions within broad issue areas like climate change or biodiversity and generally provided information on the drivers of change. The consequences of those drivers, such as impacts on society and policy responses, have tended to be left to the research community to investigate, using scenarios of drivers as inputs to their studies, producing projections of a disparate set of relevant output metrics. While this approach has had many benefits, it has fallen short of producing a robust, comparable literature describing outcomes across studies in common metrics. We argue that new scenarios are needed that extend current approaches to be organized around common outcome metrics for the well-being and resilience of society and ecosystems. We propose an approach that would focus on agreed upon outcomes for well-being and resilience as well as critical drivers of change, cut across issues and scales in multiple sectors, and draw on new systematic methods of scenario generation and discovery to highlight scenarios that are most critical in understanding societal risks and responding to them. Research derived from this outcome-based scenario development approach would facilitate improved assessment of risks of and responses to a range of stressors and the multi-sector interactions they generate.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

SMART Mobility. Modeling Workflow Development, Implementation, and Results Capstone Report

The U.S. Department of Energy’s Systems and Modeling for Accelerated Research in Transportation (SMART) Mobility Consortium is a multiyear, multi-laboratory collaborative, managed by the Energy Efficient Mobility Systems Program of the Office of Energy Efficiency and Renewable Energy, Vehicle Technologies Office, dedicated to further understanding the energy implications and opportunities of advanced mobility technologies and services. The first three-year research phase of SMART Mobility occurred from 2017 through 2019, and included five research pillars: Connected and Automated Vehicles, Mobility Decision Science, Multi-Modal Freight, Urban Science, and Advanced Fueling Infrastructure. A sixth research thrust integrated aspects of all five pillars to develop a SMART Mobility Modeling Workflow to evaluate new transportation technologies and services at scale. This report summarizes the work of the SMART Mobility Modeling Workflow effort. The SMART Mobility Modeling Workflow was developed to evaluate new transportation technologies such as connectivity, automation, sharing, and electrification through multi-level systems analysis that captures the dynamic interactions between technologies. By integrating multiple models across different levels of fidelity and scale, the Workflow yields insights about the influence of new mobility and vehicle technologies at the system level. For information about the other Pillars, please refer to the relevant pillar’s Capstone Report.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Hydropower Cyber-Physical Configurations

The U.S. Department of Energy’s Water Power Technologies Office funded Pacific Northwest National Laboratory, Argonne National Laboratory, and the National Renewable Energy Laboratory to develop a typology to characterize the variety and pervasiveness of cyber-physical configurations across the nation’s hydropower fleet. Outreach to owners and operators returned configurations for 275 hydropower plants or approximately 12% of the fleet. Components (OT and IT), systems, and connections among systems differed among plants according to function, age, position in the river cascade, and many other factors. Seven cyber-physical configuration types labeled A through I, included from 2 to dozens of plants. They were differentiated by how pervasive data and control connections were among cyber-physical components and how frequently control signals paired with data signals in a feedback loop. The flow of data and control within each type implies what cybersecurity vulnerabilities may exist, and what mitigation actions may be most effective. A self-assessment approach allows plant operators to identify the configuration type similar to their plant and link to the lessons learned and best practices information. The cyber-physical typology reinforces the idea that hydropower facilities vary widely, but it also identifies groups that highlight similarities in how their cyber-physical components interact. This helps address fleetwide cybersecurity needs by identifying a reasonable number of configuration types that share risks, vulnerabilities, and potential mitigations.

13 HYDRO ENERGY↗
DOE OSTI2023

The Field Epidemiology Training Program’s Contribution to Essential Public Health Functions in Seven National Public Health Institutes

Objective: This study explores how Field Epidemiology Training Programs (FETP) whose National Public Health Institutes (NPHI) are supported by U.S. Centers for Disease Control and Prevention (CDC) have contributed to strengthening essential public health functions. Methods: We conducted 96 semi-structured interviews with public health experts including NPHI staff, non-NPHI government staff, and staff from non-governmental and international organizations in Cambodia, Colombia, Liberia, Mozambique, Nigeria, Rwanda, and Zambia. We managed data using MAXQDA and employed direct content analysis to derive themes. Results: Three overarching themes emerged in relation to FETPs’ role within the NPHIs’ public health functions. These themes included contribution to improving country surveillance systems, role in providing leadership in outbreak responses, and strengthening countries’ and the NPHIs’ surveillance workforce capacity. Participants also shared challenges around FETPs’ implementation and suggestions for improvement. Conclusion: The results demonstrate the value of FETPs in strengthening public health systems through building workforce capacity and improving surveillance systems. By identifying the successes of FETPs in contributing to essential public health functions, our findings might inform current and future FETP implementation and its integration into NPHIs.

60 APPLIED LIFE SCIENCES↗