Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

HamLib: A Library of Hamiltonians for Benchmarking Quantum Algorithms and Hardware

For a considerable time, large datasets containing problem instances have proven valuable for analyzing computer hardware, software, and algorithms. One notable example of the value of large datasets is ImageNet [1], a vast repository of images that has been instrumental in testing numerous deep learning packages. Similarly, in the domain of computational chemistry and materials science, the availability of extensive datasets such as the Protein Data Bank [2], the Materials Project [3], and QM9 [4] has greatly facilitated the evaluation of new algorithms and software approaches, while also promoting standardization within the field. These well-defined datasets and problem instances, in turn, serve as the foundation for creating benchmarking suites like MLPerf [5] and LINPACK [6], [7]. These suites enable fair and rigorous comparisons of different methodologies and solutions, fostering continuous advancements in various areas of computer science and beyond.

Sawaya, Nicolas PD↗
DOE OSTI2024

Prompt Phrase Ordering Using Large Language Models in HPC: Evaluating Prompt Sensitivity

Large language models (LLMs) have demonstrated effective performance in domain-specific tasks, often requiring a well-designed prompt to guide their responses. However, optimizing the right prompt is challenging due to prompt sensitivity—the phenomenon where small changes in the prompt can lead to significant variations in performance. In this study, we evaluate prompt performance by examining all permutations of independent phrases to investigate prompt sensitivity and robustness. We used two datasets: the GSM8k dataset, which assesses mathematical reasoning, and a custom template prompt for summarizing database metadata. Our goal was to evaluate the performance across all permutations of a sequence of prompt phrases. The study was conducted using the llama3-instruct- 7B model hosted on Ollama, with computations parallelized in a high-performance computing environment. By comparing the average index of phrases in the best and worst-performing prompts, we found that the order of independent phrases within a prompt significantly impacts LLM performance. Additionally, we used Hamming distance to assess changes between phrase orderings, concluding that prompt modifications can dramatically affect scores, often by almost random chance. These findings support existing research on prompt sensitivity. We discuss the challenges of prompt optimization, noting that altering phrases in a successful prompt does not always result in another successful prompt.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Metabolic Costs of Walking with Weighted Vests

ABSTRACT Introduction The US Army Load Carriage Decision Aid (LCDA) metabolic model is used by militaries across the globe and is intended to predict physiological responses, specifically metabolic costs, in a wide range of dismounted warfighter operations. However, the LCDA has yet to be adapted for vest-borne load carriage, which is commonplace in tactical populations, and differs in energetic costs to backpacking and other forms of load carriage. Purpose The purpose of this study is to develop and validate a metabolic model term that accurately estimates the effect of weighted vest loads on standing and walking metabolic rate for military mission-planning and general applications. Methods Twenty healthy, physically active military-age adults (4 women, 16 men; age, 26 ± 8 yr old; height, 1.74 ± 0.09 m; body mass, 81 ± 16 kg) walked for 6 to 21 min with four levels of weighted vest loading (0 to 66% body mass) at up to 11 treadmill speeds (0.45 to 1.97 m·s −1 ). Using indirect calorimetry measurements, we derived a new model term for estimating metabolic rate when carrying vest-borne loads. Model estimates were evaluated internally byk-fold cross-validation and externally against 12 reference datasets (264 total participants). We tested if the 90% confidence interval of the mean paired difference was within equivalence limits equal to 10% of the measured walking metabolic rate. Estimation accuracy, precision, and level of agreement were also evaluated by the bias, standard deviation of paired differences, and concordance correlation coefficient (CCC), respectively. Results Metabolic rate estimates using the new weighted vest term were statistically equivalent (P< 0.01) to measured values in the current study (bias, −0.01 ± 0.54 W·kg −1 ; CCC, 0.973) as well as from the 12 reference datasets (bias, −0.16 ± 0.59 W·kg −1 ; CCC, 0.963). Conclusions The updated LCDA metabolic model calculates accurate predictions of metabolic rate when carrying heavy backpack and vest-borne loads. Tactical populations and recreational athletes that train with weighted vests can confidently use the simplified LCDA metabolic calculator provided as Supplemental Digital Content to estimate metabolic rates for work/rest guidance, training periodization, and nutritional interventions.

Sport Sciences↗
NASA NTRS2010

Statistical Analysis of Model Data for Operational Space Launch Weather Support at Kennedy Space Center and Cape Canaveral Air Force Station

The 12-km resolution North American Mesoscale (NAM) model (MesoNAM) is used by the 45th Weather Squadron (45 WS) Launch Weather Officers at Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS) to support space launch weather operations. The 45 WS tasked the Applied Meteorology Unit to conduct an objective statistics-based analysis of MesoNAM output compared to wind tower mesonet observations and then develop a an operational tool to display the results. The National Centers for Environmental Prediction began running the current version of the MesoNAM in mid-August 2006. The period of record for the dataset was 1 September 2006 - 31 January 2010. The AMU evaluated MesoNAM hourly forecasts from 0 to 84 hours based on model initialization times of 00, 06, 12 and 18 UTC. The MesoNAM forecast winds, temperature and dew point were compared to the observed values of these parameters from the sensors in the KSC/CCAFS wind tower network. The data sets were stratified by model initialization time, month and onshore/offshore flow for each wind tower. Statistics computed included bias (mean difference), standard deviation of the bias, root mean square error (RMSE) and a hypothesis test for bias = O. Twelve wind towers located in close proximity to key launch complexes were used for the statistical analysis with the sensors on the towers positioned at varying heights to include 6 ft, 30 ft, 54 ft, 60 ft, 90 ft, 162 ft, 204 ft and 230 ft depending on the launch vehicle and associated weather launch commit criteria being evaluated. These twelve wind towers support activities for the Space Shuttle (launch and landing), Delta IV, Atlas V and Falcon 9 launch vehicles. For all twelve towers, the results indicate a diurnal signal in the bias of temperature (T) and weaker but discernable diurnal signal in the bias of dewpoint temperature (T(sub d)) in the MesoNAM forecasts. Also, the standard deviation of the bias and RMSE of T, T(sub d), wind speed and wind direction indicated the model error increased with the forecast period all four parameters. The hypothesis testing uses statistics to determine the probability that a given hypothesis is true. The goal of using the hypothesis test was to determine if the model bias of any of the parameters assessed throughout the model forecast period was statistically zero. For th is dataset, if this test produced a value >= -1 .96 or <= 1.96 for a data point, then the bias at that point was effectively zero and the model forecast for that point was considered to have no error. A graphical user interface (GUI) was developed so the 45 WS would have an operational tool at their disposal that would be easy to navigate among the multiple stratifications of information to include tower locations, month, model initialization times, sensor heights and onshore/offshore flow. The AMU developed the GUI using HyperText Markup Language (HTML) so the tool could be used in most popular web browsers with computers running different operating systems such as Microsoft Windows and Linux.

Bauman, William H., III↗
DOE OSTI2022

Ultrafast Error-bounded Lossy Compression for Scientific Datasets

Today's scientific high-performance computing applications and advanced instruments are producing vast volumes of data across a wide range of domains, which impose a serious burden on data transfer and storage. Error-bounded lossy compression has been developed and widely used in the scientific community because it not only can significantly reduce the data volumes but also can strictly control the data distortion based on the user-specified error bound. Existing lossy compressors, however, cannot offer ultrafast compression speed, which is highly demanded by numerous applications or use cases (such as in-memory compression and online instrument data compression). In this paper, we propose a novel ultrafast error-bounded lossy compressor that can obtain fairly high compression performance on both CPUs and GPUs and with reasonably high compression ratios. The key contributions are threefold. (1) We propose a generic error-bounded lossy compression framework---called SZx---that achieves ultrafast performance through its novel design comprising only lightweight operations such as bitwise and addition/subtraction operations, while still keeping a high compression ratio. (2) We implement SZx on both CPUs and GPUs and optimize the performance according to their architectures. (3) We perform a comprehensive evaluation with six real-world production-level scientific datasets on both CPUs and GPUs. Experiments show that SZx is 2~16x faster than the second-fastest existing error-bounded lossy compressor (either SZ or ZFP) on CPUs and GPUs, with respect to both compression and decompression.

Yu, Xiaodong↗
NASA NTRS2021

Exploring Sentinel-1 and Sentinel-2 diversity for Flood inundation mapping using deep learning

Identification of flood water extent from satellite images has historically relied on either synthetic aperture radar (SAR) or multi-spectral (MS) imagery. MS sensors are limited to cloud free conditions, whereas SAR imagery is plagued by noise-like speckle. Prior studies that use combinations of MS and SAR data to overcome individual limitations of these sensors have not fully examined sensitivity of flood mapping performance to different combinations of SAR and MS derived spectral indices or band transformations in color space. This study explores the use of diverse bands of Sentinel 2 (S2) through well-established water indices and Sentinel 1 (S1) derived SAR imagery along with their combinations to assess their capability for generating accurate flood inundation maps. The robustness in performance of S-1 and S-2 band combinations was evaluated using 446 hand labeled flood inundation images spanning across 11 flood events from Sen1Floods11 dataset which are highly diverse in terms of land cover as well as location. A modified K-fold cross validation approach is used to evaluate the performance of 32 combinations of S1 and S2 bands using a fully connected deep convolutional neural network known as U-Net. Our results indicated that usage of elevation information has improved the capability of S1 imagery to produce more accurate flood inundation maps. Compared to a median F1 score of 0.62 when using only S1 bands, the combined use of S1 and elevation information led to an improved median F1 score of 0.73. Water extraction indices based on S2 bands have a statistically significant superior performance in comparison to S1. Among all the band combinations, HSV (Hue, Saturation, Value) transformation of S2 bands provides a median F1 score of 0.9, outperforming the commonly used water spectral indices owing to HSV’s transformation’s superior contrast distinguishing abilities. Additionally, U-Net algorithm was able to learn the relationship between raw S2 based water extraction indices and their corresponding raw S2 bands, but not of HSV owing to relatively complex computation involved in the latter. Results of the paper establishes important benchmarks for the extension of S1 and S2 data-based flood inundation mapping efforts over large spatial extents.

Goutam Konapala↗
DOE OSTI2021

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) peak intensity normalization for complex mixture analyses

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) is a preferred technique for analyzing complex organic mixtures. Currently, there is no consensus normalization approach, nor an objective method for selecting one, for quantitative analyses of FT-ICR-MS data. We investigate a method to evaluate and score the amount of bias various normalization approaches introduce into the data. We evaluate the ability of the Statistical Procedure for the Analysis of Normalization Strategies (SPANS) to guide selection of appropriate normalization approaches for two different FT-ICR-MS datasets. Further, we test the robustness of SPANS results to changes in SPANS parameter values and assess the impact of using various normalization approaches on downstream statistical analyses. The normalization approach identified by SPANS differed for the two datasets. Normalization methods impacted the statistical significance of peaks differently, underscoring the importance of carefully evaluating potential methods. More consistent SPANS scores resulted when at least 120 significant peaks are used, where larger sets of peaks were obtained by increasing the p-value threshold. Interestingly, we show that Total Sum Scaling and highest peak normalization, used in previous studies, underperformed relative to SPANS-recommended normalization approaches. While there is no single, best normalization method for all datasets, SPANS provides a mechanism to identify an appropriate normalization method for analyzing FT-ICR-MS data quantitatively. As a result, the number of peaks used in the background distributions of SPANS contributes more significantly to the reproducibility of results than the p-value thresholds used to obtain those peaks.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Evaluation of the Self Retrieval Augmented Generation Technique on Common Security Advisory Framework Data

This small experimental report evaluates a variation of Retrieval Augmented Generation (RAG), called Self-RAG. This method uses a generative language model that incorporates retrieved facts into its generation and is explicitly trained to be able to determine whether retrieved information is enough to answer the input query, with a user-defined threshold for confidence. We performed an experiment using data from the publicly available CISA Common Security Advisory Framework (CSAF) repository (https://github.com/cisagov/CSAF) as the database of facts to be used in retrieval. Qualitative results from the experiment demonstrate that the Self-RAG method has some ability to provide reasonable answers to queries that are in the dataset and will often ignore irrelevant information when asked outside of domain questions (e.g., general facts). In settings with deliberately confusing questions (the question is within domain, but asks about a fabricated advisory), it was able to refuse 40% of the time without further adjustments to the original framework. While this performance is not sufficient for current practical use, further improvements to data formatting, disambiguating results, and leveraging threshold values could improve performance significantly. However, evaluating this will require more extensive evaluations on larger datasets and potentially better models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Model bias in simulating major chemical components of PM 2.5 in China

High concentrations of PM 2.5 (particulate matter with an aerodynamic diameter less than 2.5 µm) in China have caused severe visibility degradation. Accurate simulations of PM 2.5 and its chemical components are essential for evaluating the effectiveness of pollution control strategies and the health and climate impacts of air pollution. In this study, we compared the GEOS-Chem model simulations with comprehensive datasets for organic aerosol (OA), sulfate, nitrate, and ammonium in China. Model results are evaluated spatially and temporally against observations. The new OA scheme with a simplified secondary organic aerosol (SOA) parameterization significantly improves the OA simulations in polluted urban areas, highlighting the important contributions of anthropogenic SOA from semivolatile and intermediate-volatility organic compounds. The model underestimates sulfate and overestimates nitrate for most of the sites throughout the year. More significant underestimation of sulfate occurs in winter, while the overestimation of nitrate is extremely large in summer. The model is unable to capture some of the main features in the diurnal pattern of the PM 2.5 chemical components, suggesting inaccuracies in the presented processes. Potential model adjustments that may lead to a better representation of the boundary layer height, the precursor emissions, hydroxyl radical concentrations, the heterogeneous formation of sulfate and nitrate, and the wet deposition of nitric acid and nitrate have been tested in the sensitivity analysis. The results show that uncertainties in chemistry perhaps dominate the model biases. The proper implementation of heterogeneous sulfate formation and the good estimates of the concentrations of sulfur dioxide, hydroxyl radical, and aerosol liquid water are essential for the improvement of the sulfate simulation. The update of the heterogeneous uptake coefficient of nitrogen dioxide significantly reduces the modeled concentrations of nitrate. However, the large overestimation of nitrate concentrations remains in summer for all tested cases. The possible bias in the chemical production and the wet deposition of nitrate cannot fully explain the model overestimation of nitrate, suggesting issues related to the atmospheric removal of nitric acid and nitrate. A better understanding of the atmospheric nitrogen budget, in particular, the role of the photolysis of particulate nitrate, is needed for future model developments. Moreover, the results suggest that the remaining underestimation of OA in the model is associated with the underrepresented production of SOA.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Macro-physical Properties of Shallow Cumulus from Integrated ARM Observations (Final Report)

Fair-weather shallow cumuli (ShCu) play an important role in many climate-related processes. Irregular geometry of ShCu and their strong temporal and spatial variability make it challenging to observe ShCu holistically and to represent them correctly in climate models. To improve ShCu parameterizations, information on both vertically and horizontally resolved cloud properties is required. Commonly, the vertically resolved cloud properties are provided by zenith pointing lidar-radar observations with a very narrow field of view (FOV). Thus, these “pencil-beam” properties may not be representative of a larger surrounding area. Limited number of areal-averaged cloud properties, such as fractional sky cover (FSC), are offered typically by wide-FOV observations. The main goal of our project was to integrate advantages of the narrow-FOV (vertical structure of clouds) and wide-FOV (spatial arrangement of clouds) observations for an improved characterization of single-layer ShCu observed at the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) site for an 18-yr period (2000-2017). There are four major accomplishments of our project, First, an updated operational cloud classification for days with ShCu has been suggested and evaluated through a detailed comparison with the manually curated records. Our classification extends successfully the latest ARM cloud type Value Added Product (VAP) based on the Active Remote Sensing of Clouds (ARSCL) cloud product by incorporating both cloud fraction (CF) provided by narrow-FOV ceilometer data and FSC from wide-FOV images offered by a Total Sky Imager (TSI). Moreover, our classification allows one to identify impact of instrumentation changes at the SGP site, namely the transition to KAZRARSCL with the updated cloud radar, on the identification of periods with single-layer ShCu. Second, a new approach that resolves cloud area distributions for a given region (up to 4x4 km 2 ) has been suggested and cloud equivalent diameters (CEDs) have been estimated for the first time. These estimations have been performed over a wide range of cloud sizes (about 0.01–3.5 km) with high temporal resolution (30s) using wide-FOV TSI images and cloud base height (CBH) provided by complementary narrow-FOV lidar measurements. Our simple and computationally inexpensive approach offers a previously unavailable dataset for process studies in the convective boundary layer and evaluation of ShCu parameterizations in cloud-resolving models. Third, a long-term integrated record of ShCu macrophysical properties has been developed. The developed record represents the longest available compilation of events with ShCu and includes (i) a novel visualization of the spatial variability in cloud cover both along- and across-wind directions, (ii) updated estimates of narrow-FOV CF and wide-FOV FSC, (iii) updated narrow-FOV CBH, and (iv) complementary data, such as wind speed and direction from the 915-MHz Radar Wind Profiler (RWP) data. The developed record has been used successfully to assess conventional observational estimates of cloud cover and their sensitivity to the following two factors: (i) instrument-dependent cloud detection and data merging criteria and (ii) FOV configuration. Fourth, co-variability of the ShCu macrophysical properties and environmental parameters has been analyzed for a 3-yr period (2016-2018). Our initial analysis includes diurnal changes of FSCs obtained for clouds with small, moderate and large CEDs and several environmental parameters, such as lifted condensation level (LCL) and mixed layer height (zi). Preliminary results of our analysis suggest that the horizontal extent of ShCu is controlled substantially by the sign and magnitude of difference between these two parameters (zi-LCL): the CED tends to grow with increase of this difference (zi exceeds LCL). We have initiated relationships between the ShCu and key atmospheric parameters that control both the development and evolution of ShCu using our new data product, which combines effectively the advantages of narrow-FOV data offered by zenith pointing cloud radars and lidars and wide-FOV TSI images. While the latest instrumentation at the ARM sites may address these challenging relationships in the future, we believe that the historical ARM data at the SGP site has not yet been fully utilized. Overall, our data product can be used by researchers working on a wide range of climate-related projects. These projects may include (i) a comprehensive evaluation of outputs from the Large-Eddy Simulation (LES) and single-column models for their future improvement, (ii) the representativeness of “short-period” results obtained from the previous model and observational studies and (iii) the planning of future field campaigns with focus on improved understanding of the diurnal cycle of cumulus convection.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Lab Homes

This dataset includes processed data from the Lab Homes (LH) Test Facility located on the PNNL campus in Richland, WA. This a set of 2 identical homes that allow for the side-by-side comparison/performance evaluation of different technologies under the same weather at any given time. The dataset spans December 6, 2021 to December 27, 2021 and represents a series of tests performed; calibration, set-point excitation, pre-heating, free-floating and warm up. The measurements correspond to whole building electrical power, HVAC energy use, water heating, appliances and lighting, as well as space temperatures, space humidity, window glass surface temperatures, through glass solar radiation, and meterological data from an onsite meteorological weather station. In addition to the measurements, a metadata .json file, a .ttl file to visualize the data as per BRICK schema, and a detailed .pdf description of the dataset are also provided.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Systematic and objective evaluation of Earth system models: PCMDI Metrics Package (PMP) version 3

Systematic, routine, and comprehensive evaluation of Earth system models (ESMs) facilitates benchmarking improvement across model generations and identifying the strengths and weaknesses of different model configurations. By gauging the consistency between models and observations, this endeavor is becoming increasingly necessary to objectively synthesize the thousands of simulations contributed to the Coupled Model Intercomparison Project (CMIP) to date. The Program for Climate Model Diagnosis and Intercomparison (PCMDI) Metrics Package (PMP) is an open-source Python software package that provides quick-look objective comparisons of ESMs with one another and with observations. The comparisons include metrics of large- to global-scale climatologies, tropical inter-annual and intra-seasonal variability modes such as the El Niño–Southern Oscillation (ENSO) and Madden–Julian Oscillation (MJO), extratropical modes of variability, regional monsoons, cloud radiative feedbacks, and high-frequency characteristics of simulated precipitation, including its extremes. The PMP comparison results are produced using all model simulations contributed to CMIP6 and earlier CMIP phases. An important objective of the PMP is to document the performance of ESMs participating in the recent phases of CMIP, together with providing version-controlled information for all datasets, software packages, and analysis codes being used in the evaluation process. Among other purposes, this also enables modeling groups to assess performance changes during the ESM development cycle in the context of the error distribution of the multi-model ensemble. Quantitative model evaluation provided by the PMP can assist modelers in their development priorities. In this paper, we provide an overview of the PMP, including its latest capabilities, and discuss its future direction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Industrial control system device classification using network traffic features and neural network embeddings

Characterization of modern cyber–physical Industrial Control System (ICS) devices is critical to the evaluation of their security posture and an understanding of the underlying industrial processes with which they interact. In this work, we address two related ICS device identification tasks: (1) separating ICS from non-ICS devices and (2) identifying specific ICS device types. We propose two distinct methods (one based on the existing IP2Vec method, and a novel traffic-features-based method) for achieving the first task. For transferability of the first task between two datasets, the traffic-features-based method performs significantly better (75% overall accuracy) compared to IP2Vec (22.5% overall accuracy). We further propose a novel method called DNP2Vec to address the second task. DNP2Vec is evaluated on two different datasets and achieves perfect multi-class classification accuracy (100%) for both datasets.

42 ENGINEERING↗
DOE OSTI2024

Ecological Benchmark for Radionuclides

The Ecological Benchmark Tool for radionuclides dataset serves as a comprehensive repository of benchmarks designed to assess ecological risks at contaminated sites. This tool facilitates the evaluation of various environmental media and contaminants, supporting regulatory compliance and ecological protection. Benchmarks are available for sediment, soil, and surface water. The dataset also provides species-specific benchmarks for fish, plants, birds, mammals, and invertebrates. Users can select benchmark sources, media, individual radionuclides, and retrieve results in tabular or spreadsheet formats for analysis. Benchmarks are derived from authoritative sources, including government agencies, scientific councils, and academic publications. The dataset supports ecological risk assessments, regulatory decision-making, and environmental planning, with tools for benchmarking against radiological thresholds, sensitive species protection, and habitat impact evaluations. This structured approach ensures a robust evaluation of ecological risks tailored to site-specific and regulatory needs.

Stewart, Debra [Oak Ridge National Laboratory (ORN↗
DOE OSTI2024

Data Driven Commercial Building Energy Code Compliance and Technology Inventory for New York City

Building Performance Standards (BPS) are gaining national traction. A BPS will require new processes in the design, construction, and operation of buildings that take the occupants into account and enable predictive analysis to ensure compliance with current and future GHG emissions caps. In New York City, most buildings over 25,000 square feet will be regulated by a BPS starting in 2024, regardless of whether it is new construction permitted under current energy codes or an existing building. This research is one of the first to begin the evaluation of a long-term series of building policies in the context of an open data ecosystem, in cooperation with city agencies. Existing building policies enacted in NYC have ranged from building energy benchmarking and labeling to energy audits to the regulation of GHG emission in buildings. Through the development of a dataset related to building technologies and energy consumption, this project can help to evaluate if meaningful conclusions can be drawn for the data that has been largely self-reported in compliance with city regulations. This project will also provide lessons learned from a deep dive into these types of datasets to provide best practices for municipalities or states seeking to embark on policies like those enacted in NYC. In addition, a Building Automation System (BAS) Stretch Standard of Care (SSOC) for owners, designers, and building operators will enable the measurement and predictive analysis of energy consumption and GHG emissions at the plant, system, or component level, in anticipation of regulated GHG limits on buildings based on energy use. The SSOC is expected to be suitable for use on a national level. The primary feature of an SSOC is a standardized format for a set of BAS points that can be used to control and to gather data from individual plants, systems, or components that are related to building energy consumption. This project examined how measurements compare to prescriptive or simulation-based energy code targets, finding little correlation between predictive 8760-hour energy modeling and actual energy consumption for a small sample (n=27) of buildings constructed after 2015. Other analysis found that, while large multifamily housing (MFH) buildings showed a general trend similar to predicted reductions in energy use from the implementation of model commercial energy codes, this trend was not evident in the office, K-12 school, and hotel use groups in NYC. No upward or downward trends in energy consumption were found when buildings were grouped by size. Energy audit data were analyzed and it appears that there is bias by audit company on measures recommended to clients. Further research should be performed to cross-analyze this with other attributes, such as building size, vintage, and number of stories. Analysis found that for 281 buildings that were permitted and completed after 2015 and had submitted benchmarking data in 2022, between 81% and 96% (by use group) were found to be in compliance with the 2024 to 2029 NYC BPS emission caps, and between 55% and 89% were in compliance with the 2030-2034 caps. This work is beneficial to the public in helping policymakers and building stakeholders better understand the wide-ranging implications of a BPS.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Low-Temperature Geothermal Play Fairway Analysis for the Denver Basin: Preprint

This project is part of a national initiative to showcase the benefits of incorporating low-temperature geothermal resource assessment into the deployment of geothermal heating and cooling (GHC), combined heat and power (CHP), and geothermal direct use (GDU) technologies. The initiative was established to accelerate the country's decarbonization efforts by identifying potential for low-temperature geothermal resource utilization (< 150 Degrees Celsius, i.e., GHC, CHP, and GDU) in selected sedimentary basins with numerous population centers. The Play Fairway Analysis (PFA) methodologies in this study were adapted from previous PFA investigations of sedimentary basin geothermal play types (SBGPTs) that evaluated the potential for low-temperature resources (< 150 Degrees Celsius). Workflows, relevant datasets, python code, common and composite geological criteria maps are utilized to develop low-temperature geothermal resource favorability maps for the Denver Basin, a sedimentary basin spanning Colorado, Nebraska, and Wyoming. The replication of these methodologies in other SBGPTs can evaluate potential for low-temperature resources. To facilitate future assessment of low-temperature geothermal resources in SBGPTs, this project provides PFA workflows, data, tools, and favorability maps that will ultimately support the utilization of low-temperature geothermal resources in sedimentary basins.

Denver Basin↗
DOE OSTI2024

A Million Person Study Innovation: Evaluating Cognitive Impairment and other Morbidity Outcomes from Chronic Radiation Exposure Through Linkages with the Centers for Medicaid and Medicare Services Assessment and Claims Data

Here, the study of One Million U.S. Radiation Workers and Veterans, the Million Person Study (MPS), examines the health consequences, both cancer and non-cancer, of exposure to ionizing radiation received gradually over time. Recently the MPS has focused on mortality patterns from neurological and behavioral conditions, e.g., Parkinson's disease, Alzheimer's disease, dementia, and motor neuron disease such as amyotrophic lateral sclerosis. A fuller picture of radiation-related late effects comes from studying both mortality and the occurrence (incidence) of conditions not leading to death. Accordingly, the MPS is identifying neurocognitive diagnoses from fee-for-service insurance claims from the Centers for Medicare and Medicaid Services (CMS), among Medicare beneficiaries beginning in 1999 (the earliest date claims data are available). Linkages to date have identified ∼540,000 workers with available health information. Such linkages provide individual information on important co-factor and confounding variables such as smoking, alcohol consumption, blood pressure, obesity, diabetes and many other health and demographic characteristics. The total person-level set of time-dependent variables, outcomes, organ-specific dose measures, co-factors, and demographics will be massive and much too large to be evaluated with standard software. Thus, development of specialized open-source software designed for large datasets (Colossus) is nearly complete. The wealth of information available from CMS claims data, coupled with individual dose reconstructions, will thus greatly enhance the quality and precision of health evaluations for this new field of low-dose radiation and neurocognitive effects.

Dauer, Lawrence T.↗
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗