Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “compiler verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Automated Test for NASA CFS

The core Flight System (cFS) is a flight software (FSW) product line developed by the Flight Software Systems Branch (FSSB) at NASA's Goddard Space Flight Center (GSFC). The cFS uses compile-time configuration parameters to implement variable requirements to enable portability across embedded computing platforms and to implement different end-user functional needs. The verification and validation of these requirements is proving to be a significant challenge. This paper describes the challenges facing the cFS and the results of a pilot effort to apply EXB Solution's testing approach to the cFS applications.

Flight software↗

SPEL: Software tool for Porting E3SM Land Model with OpenACC in a Function Unit Test Framework

Most high-end computers adopt hybrid architecture, porting a large-scale scientific code onto accelerators is necessary. The paper presents a generic method for porting large-scale scientific code onto accelerators using compiler directives within a modularized function unit test platform. We have implemented the method and designed a software tool (SPEL) to port the E3SM Land Model (ELM) onto the GPUs in the Summit computer. SPEL automatically generates GPU-ready test modules for all ELM functions, such as CanopyFlux, SoilTemperature, and EcosystemDynamics. SPEL breaks the ELM into a collection of standalone unit test programs for easy code verification and further performance improvement. We further optimize several ELM test modules with advanced techniques, including memory reduction, reconstructed parallel loops, and asynchronous GPU kernel launch. We hope our study will inspire new toolkit developments that expedite large-scale scientific code porting with compiler directives.

Schwartz, Peter↗

Proving Correctness for Pointer Programs in a Verifying Compiler

This research describes a component-based approach to proving the correctness of programs involving pointer behavior. The approach supports modular reasoning and is designed to be used within the larger context of a verifying compiler. The approach consists of two parts. When a system component requires the direct manipulation of pointer operations in its implementation, we implement it using a built-in component specifically designed to capture the functional and performance behavior of pointers. When a system component requires pointer behavior via a linked data structure, we ensure that the complexities of the pointer operations are encapsulated within the data structure and are hidden to the client component. In this way, programs that rely on pointers can be verified modularly, without requiring special rules for pointers. The ultimate objective of a verifying compiler is to prove-with as little human intervention as possible-that proposed program code is correct with respect to a full behavioral specification. Full verification for software is especially important for an agency like NASA that is routinely involved in the development of mission critical systems.

Kulczycki, Gregory↗

Integrated Medical Model Verification, Validation, and Credibility

The Integrated Medical Model (IMM) was designed to forecast relative changes for a specified set of crew health and mission success risk metrics by using a probabilistic (stochastic process) model based on historical data, cohort data, and subject matter expert opinion. A probabilistic approach is taken since exact (deterministic) results would not appropriately reflect the uncertainty in the IMM inputs. Once the IMM was conceptualized, a plan was needed to rigorously assess input information, framework and code, and output results of the IMM, and ensure that end user requests and requirements were considered during all stages of model development and implementation. METHODS: In 2008, the IMM team developed a comprehensive verification and validation (VV) plan, which specified internal and external review criteria encompassing 1) verification of data and IMM structure to ensure proper implementation of the IMM, 2) several validation techniques to confirm that the simulation capability of the IMM appropriately represents occurrences and consequences of medical conditions during space missions, and 3) credibility processes to develop user confidence in the information derived from the IMM. When the NASA-STD-7009 (7009) was published, the IMM team updated their verification, validation, and credibility (VVC) project plan to meet 7009 requirements and include 7009 tools in reporting VVC status of the IMM. RESULTS: IMM VVC updates are compiled recurrently and include 7009 Compliance and Credibility matrices, IMM VV Plan status, and a synopsis of any changes or updates to the IMM during the reporting period. Reporting tools have evolved over the lifetime of the IMM project to better communicate VVC status. This has included refining original 7009 methodology with augmentation from the NASA-STD-7009 Guidance Document. End user requests and requirements are being satisfied as evidenced by ISS Program acceptance of IMM risk forecasts, transition to an operational model and simulation tool, and completion of service requests from a broad end user consortium including Operations, Science and Technology Planning, and Exploration Planning. CONCLUSIONS: The VVC approach established by the IMM project of combining the IMM VV Plan with 7009 requirements is comprehensive and includes the involvement of end users at every stage in IMM evolution. Methods and techniques used to quantify the VVC status of the IMM have not only received approval from the local NASA community but have also garnered recognition by other federal agencies seeking to develop similar guidelines in the medical modeling community.

Validation↗

Water at the Research and Education Complex [Poster]

Purpose: To identify the management methods for stormwater, sanitary sewer, and potable water throughout the facilities at the Research and Education Complex (REC) using drawings and written documents. To confirm the locations and status of wells located throughout the REC. To compile appropriate documentation that relates to water for each facility at the REC, verify that documentation is accurate, and create any new documentation that is necessary through field verification.

54 ENVIRONMENTAL SCIENCES↗

Code Migration and Optimization for Calculating Atmospheric Attenuation in RF Link Budgets Within Glenn Research Center Communication Analysis Suite

This technical report encompasses development completed on the Glenn Research Center Communication Analysis Suite (GCAS) as it relates to converting code from the original MATLAB® (The MathWorks, Inc.) GCAS code base to C++. It focuses on a majority of the functions, structures, and intermediate code developed for radiofrequency (RF) links calculations. The report serves as a comprehensive record of the current progress and outlines the technical challenges overcome during this research. Throughout this project, numerous functions and structures were migrated end to end, with validation test cases conducted in accordance with NASA’s verification and validation efforts for standard models and simulations. This report also provides high-level documentation of the completed work; detailed low-level documentation has been compiled to provide a line-by-line explanation in alignment with NASA’s goals. This development has largely been focused on atmospheric effects on calculating RF link budgets for both terrestrial and space links. Some developments outside of atmospheric functions are also covered throughout this report; these are largely related to mathematical operations and MATLAB® structure conversions.

Brady M. Phelps↗

Active Control Evaluation for Spacecraft (ACES)

The Air Force goal is to develop vibration control techniques for large flexible spacecraft by addressing sensor, actuator, and control hardware and dynamic testing. The Active Control Evaluation for Spacecraft (ACES) program will address the Air Force goal by looking at two leading control techniques and implementing them on a structural model of a flexible spacecraft under laboratory testing. The first phase in the ACES program is to review and to assess the High Authority Control/Low Authority Control (HAC/LAC) and Filter accomodated Model Error Sensitivity Suppression (FAMESS) control techniques for testing on the modified VCOSS structure. Appropriate sensors and actuators will be available for use with both techniques; locations will be the same for both techniques. The control actuators will be positioned at the midpoint and free end of the structure. The laser source for the optical sensor is mounted on the feed mast. The beam will be reflected from a mirror on the offset antenna onto the detectors mounted above the shaker table bay. The next phase is to develop an analysis simulation with the control algorithms implemented for dynamics verification. The third phase is to convert the control laws into high level computer language and test them in the NASA-MSFC facility. The final phase is to compile all analytical and test results for performance comparisons.

Pearson, J.↗

Towards High-Assurance High-Performance Program Synthesis

Domain-specific automatic program synthesis tools, also called application generators, are playing an ever-increasing role in software development. However, high-performance application generators require difficult manual construction, and are very difficult to verify correct. This paper describes research and an implemented system that transforms program synthesis tools based on deductive synthesis into high-performance application generators. Deductive synthesis uses theorem-proving to construct solutions when given problem specifications. The verification condition for a deductive synthesis tool is essentially the soundness of the implemented inference rules. Theory Operationalization for Program Synthesis (TOPS) synergistically combines reformulation, automated mathematical classification, and compilation through partial deduction to decision procedures. It transforms general-purpose deductive synthesis, with exponential performance, into efficient special-purpose deductive synthesis, with near-linear performance. This paper describes our experience with and empirical results of PD(TH) theory-based partial deduction - in which partial deduction of a set of first-order formulae is performed within the context of a background theory. The implemented TOPS system currently performs a special variant of PD(TH) in which the compilation process results in the transformation of a set of first order formulae into the theory of an instantiated library decision procedure augmented by a compiled unit theory.

Lowry, Michael↗

Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices

Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.

Zavordsky, Bradley↗

Development and Implementation of Dynamic Scripts to Support Local Model Verification at National Weather Service Weather Forecast Offices

Local modeling with a customized configuration is conducted at National Weather Service (NWS) Weather Forecast Offices (WFOs) to produce high-resolution numerical forecasts that can better simulate local weather phenomena and complement larger scale global and regional models. The advent of the Environmental Modeling System (EMS), which provides a pre-compiled version of the Weather Research and Forecasting (WRF) model and wrapper Perl scripts, has enabled forecasters to easily configure and execute the WRF model on local workstations. NWS WFOs often use EMS output to help in forecasting highly localized, mesoscale features such as convective initiation, the timing and inland extent of lake effect snow bands, lake and sea breezes, and topographically-modified winds. However, quantitatively evaluating model performance to determine errors and biases still proves to be one of the challenges in running a local model. Developed at the National Center for Atmospheric Research (NCAR), the Model Evaluation Tools (MET) verification software makes performing these types of quantitative analyses easier, but operational forecasters do not generally have time to familiarize themselves with navigating the sometimes complex configurations associated with the MET tools. To assist forecasters in running a subset of MET programs and capabilities, the Short-term Prediction Research and Transition (SPoRT) Center has developed and transitioned a set of dynamic, easily configurable Perl scripts to collaborating NWS WFOs. The objective of these scripts is to provide SPoRT collaborating partners in the NWS with the ability to evaluate the skill of their local EMS model runs in near real time with little prior knowledge of the MET package. The ultimate goal is to make these verification scripts available to the broader NWS community in a future version of the EMS software. This paper provides an overview of the SPoRT MET scripts, instructions for how the scripts are run, and example use cases.

Zavodsky, Bradley↗

Updated Analysis of the Trash Compaction and Processing System: Water Collected from Trash, Evaluation of the Source Contaminant Control System, and Zotek® F30 Foam Processing

The Trash Compaction and Processing System (TCPS) aims to reduce volume, biologically safen, physically stabilize, manage effluents, and recover resources from astronaut trash in the International Space Station (ISS). At NASA Ames Research Center (ARC), there are two test systems: the Heat Melt Compactor (HMC), which compacts the trash into a tile with temperature and pressure, and the Source Contaminant Control System (SCCS), which is the gas management subsystem aimed to reduce contaminants by converting them into carbon dioxide (CO 2 ) and water (H 2 O) vapor. Both the HMC and SCCS serve as test systems that support TCPS development by conducting risk reduction activities associated with an ISS technology demonstration. In this paper, the risk reduction activities discussed are the testing of different trash models for operational scenarios in the HMC, investigating the water extracted from those trash models, and managing the gaseous effluents from the trash via contaminant conversion through the SCCS. The HMC has undergone several tests of different trash models and cases in which a containment bag is used or not. For the first topic, this paper will analyze the water collected and recovered from these tests and analyze the effect of specific trash containment bags. For the second topic, this paper will individually evaluate the carbon bed and catalytic oxidizer (CatOx) to understand what components the carbon bed removes and what contaminants are converted in the CatOx. For the third and final topic, this paper will discuss Zotek® F30 foam processing in the HMC unit and testing verification in correlation to defined TCPS requirements. A foam model was added to reduce the volume of foam that take up space in the ISS. Overall, this paper is a compilation and an updated analysis of tests conducted in the lab-scale TCPS at ARC within the past year.

TCPS↗

Code Coverage Status of the ARC Code DASSH-F

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DASSH-F serves as a steady state thermal hydraulic capability within the ARC system and replaces the SE2-ANL software that preceded it. This document identifies the set of test problems used to assess the code coverage for DASSH-F. The goal is to document what parts of the existing DASSH-F code are touched by the set of test problems and which are not. Because the verification work remains to be done on DASSH-F, one can assume that most of these issues will be resolved as part of that work. The code coverage analysis of DASSH-F was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DASSH-F. The code coverage tables are given for each submodule of DASSH-F. Because DASSH-F links to modules in DIF3D, some details on coverage changes to the DIF3D linked files is provided. As will be seen, most of the uncovered parts/files can be ignored because they are either for error message and debugging output or obviously not needed by DASSH-F today. Seven features of the DASSH-F code were identified to not be covered by the existing testing suite and thus additional verification test problems are suggested to fully cover these sections.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Electron/proton spectrometer certification documentation analyses

A compilation of analyses generated during the development of the electron-proton spectrometer for the Skylab program is presented. The data documents the analyses required by the electron-proton spectrometer verification plan. The verification plan was generated to satisfy the ancillary hardware requirements of the Apollo Applications program. The certification of the spectrometer requires that various tests, inspections, and analyses be documented, approved, and accepted by reliability and quality control personnel of the spectrometer development program.

Gleeson, P.↗

Design of optical mirror structures

The structural requirements for large optical telescope mirrors was studied with a particular emphasis placed on the three-meter Large Space Telescope primary mirror. Analysis approaches through finite element methods were evaluated with the testing and verification of a number of element types suitable for particular mirror loadings and configurations. The environmental conditions that a mirror will experience were defined and a candidate list of suitable mirror materials with their properties compiled. The relation of the mirror mechanical behavior to the optical performance is discussed and a number of suitable design criteria are proposed and implemented. A general outline of a systematic method to obtain the best structure for the three-meter diffraction-limited system is outlined. Finite element programs, using the STRUDL 2 analysis system, were written for specific mirror structures encompassing all types of active and passive mirror designs. Parametric studies on support locations, effects of shear deformation, diameter to thickness ratios, lightweight and sandwich mirror configurations, and thin shell active mirror needs were performed.

Soosaar, K.↗

Code Coverage Status of the ARC Code RCT

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. REBUS plays a pivotal role in the ARC system as the primary fuel cycle analysis capability for fast reactor problems. Over its 60 year history, ARC software usage with REBUS has been applied to numerous fast and thermal spectrum reactor analysis projects with good to excellent comparison against experiments. The RCT code is a later addition and uses the REBUS restart files to define its input. The RCT code was built to provide pin depletion details on EBR-II models and thus many features of RCT were specifically tailored to the needs of EBR-II models. Additional approximations were invoked which are likely only valid for the EBR-II reactor and the particular fuel management that was done for it. The purpose of the present work is to identify a set of test problems for RCT and assess the code coverage for those test problems. The goal is to document what parts of the existing RCT code are touched by the set of test problems and which are not. Because no detailed verification work has been done on RCT, the existing regression testing suite was chosen for the code coverage assessment. The code coverage analysis of RCT was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of RCT. The detailed coverage tables are given for each part of RCT. As will be discussed and shown, some parts of the RCT capability that are known to be used by the EBR-II analysis work are not tested by the regression testing suite. These aspects should be resolved before major source code changes are taken for the RCT software. Because REBUS and DIF3D are not subroutines of RCT, the coverage changes in both of those codes is not altered by RCT. The same is true for all of the modules of DIF3D that are used by RCT such as SYSLIB and SEGLIB.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued through 2024 and 2025. The continuous integration testing was extended to RCT, DASSH, and SE2ANL. Minor changes were required to the original continuous integration methodology to make this happen. When full confidence in the methodology is complete, a report will be created to detail the automated regression testing methodology and minor reports will be created to detail the tolerance settings that have been applied to the output for each ARC code. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The DASSH, SE2ANL, and SE2RCT manuals were completed this fiscal year. A review of the SE2ANL software identified that it is unrealistic to include updated correlations or different geometry models and it was scheduled for deprecation in favor of DASSH. The SE2ANL manual is essential for SE2RCT as they are similar but quite different in purpose. The only piece of software missing a manual consistent with the source code is NUBOW-3D which is a focus of the coming year. The code coverage report for DIF3D was updated and code coverage reports were created for REBUS, RCT, PERSENT, GAMSRC, and DASSH. Minor coverage issues were identified for all of these pieces of software which did not prevent the work done to transition them to the OneAPI compiler. Because SE2ANL was scheduled for deprecation, it was not transitioned, but it was successfully tested with the OneAPI compiler. This leaves SE2RCT and NUBOW-3D as the only pieces of software not transitioned to OneAPI and further work is required to get SE2RCT to work properly. The SE2RCT software transition will begin early next year while the NUBOW-3D software requires a manual before it can begin. Software verification work has been completed for DIF3D, REBUS, GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The PERSENT software verification work was completed this year which was somewhat delayed because of unexpected bugs in the software. The PERSENT manual was updated to detail some of the issues and discuss the bowing reactivity worth feature added in the previous fiscal year. The RCT, DASSH, SE2RCT, and NUBOW-3D software are the only maintained pieces of software without verification reports. The software verification work for DASSH will be a focus in the upcoming fiscal year and it is hoped that some of the test cases created can serve as verification tests for SE2RCT. The NUBOW-3D work will begin when the manual and requirements report are completed. Only minor industry partner software development funds were provided this year. The DASSH software was updated to handle general axial geometry for each assembly and the NUBOW-3D software was updated to incorporate a new input format and better output. Overall progress on resolving the QA gaps has been good this year.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued through 2024 and 2025. The continuous integration testing was extended to RCT, DASSH, and SE2ANL. Minor changes were required to the original continuous integration methodology to make this happen. When full confidence in the methodology is complete, a report will be created to detail the automated regression testing methodology and minor reports will be created to detail the tolerance settings that have been applied to the output for each ARC code. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The DASSH, SE2ANL, and SE2RCT manuals were completed this fiscal year. A review of the SE2ANL software identified that it is unrealistic to include updated correlations or different geometry models and it was scheduled for deprecation in favor of DASSH. The SE2ANL manual is essential for SE2RCT as they are similar but quite different in purpose. The only piece of software missing a manual consistent with the source code is NUBOW-3D which is a focus of the coming year. The code coverage report for DIF3D was updated and code coverage reports were created for REBUS, RCT, PERSENT, GAMSRC, and DASSH. Minor coverage issues were identified for all of these pieces of software which did not prevent the work done to transition them to the OneAPI compiler. Because SE2ANL was scheduled for deprecation, it was not transitioned, but it was successfully tested with the OneAPI compiler. This leaves SE2RCT and NUBOW-3D as the only pieces of software not transitioned to OneAPI and further work is required to get SE2RCT to work properly. The SE2RCT software transition will begin early next year while the NUBOW-3D software requires a manual before it can begin. Software verification work has been completed for DIF3D, REBUS, GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The PERSENT software verification work was completed this year which was somewhat delayed because of unexpected bugs in the software. The PERSENT manual was updated to detail some of the issues and discuss the bowing reactivity worth feature added in the previous fiscal year. The RCT, DASSH, SE2RCT, and NUBOW-3D software are the only maintained pieces of software without verification reports. The software verification work for DASSH will be a focus in the upcoming fiscal year and it is hoped that some of the test cases created can serve as verification tests for SE2RCT. The NUBOW-3D work will begin when the manual and requirements report are completed. Only minor industry partner software development funds were provided this year. The DASSH software was updated to handle general axial geometry for each assembly and the NUBOW-3D software was updated to incorporate a new input format and better output. Overall progress on resolving the QA gaps has been good this year.

97 MATHEMATICS AND COMPUTING↗

Reported Energy and Cost Savings from the DOE ESPC IDIQ Program: FY 2024

Energy Savings Performance Contracts (ESPCs)are a contractual mechanism that allow a federal agency to procure energy savings and facility improvements without upfront capital costs to reduce costs and resiliency. ESPCs are covered under FAR Part 23.2, and 42 USC § 8287. Section 8287(a)(2)(A) of Title 42 of the U.S. Code requires that each energy savings performance contract (ESPC) undergo an annual energy audit, resulting in a separate audit report for every project. The objective of the present report is to compile and analyze all annual ESPC audit reports issued between October 1, 2023, and September 30, 2024, for projects awarded under Generations 1, 2, and 3 of DOE’s ESPC IDIQ contracts. During this period, 205 measurement and verification (M&V) reports were produced for 200 projects; the total number of reports exceeds the number of projects because some projects generated more than one report(for example, a few projects measure savings twice per year and produce two audit reports annually, each covering a different six-month period). By aggregating the results from these individual audits, the report determines the portfolio-wide realization rate of energy and cost savings for all active ESPC projects awarded under DOE’s IDIQ program. For all 205audit reports, sufficient information was available to compare project-level estimated, reported, and guaranteed cost savings. Reported cost savings accounted for ESCO verified savings per each project’s M&V plan. The total reported cost savings for the period addressed were $\$$647.8million,compared with the total guaranteed cost savings of $\$$601.6million. On average across the reported projects: •ESPC contractors guaranteed 92.8% of the estimated cost savings• projects reported achieving 100.0% of the estimated cost savings• projects reported achieving 107.7% of the guaranteed cost savings. The M&V performed for the period indicated adjustments for government operations and maintenance impacts to savings amount to$\$$43.9millionandcould be restored with the original operational parameters for impacted projects. Accounting for this potential cost savings impact, these projects still realized 100.4% of the guaranteed cost savings. The information on estimated and reported energy savings was collected and compared for all 205of the reports examined. Based on site energy, estimated savings totaled 14.88million MMBtu, and reported savings were 15.33million MMBtu; 3.1% greater than the estimated energy savings. All of the reports examined contained sufficient information to calculate source energy savings. Based on site-adjusted source energy, total estimated energy savings were 20.90 million MMBtu, and reported savings were 21.22million MMBtu, 101.5% of the estimated energy savings. For water savings, the estimated savings were 11,539,055 kGal and the reported savings were 13,315,930 kGal. This means 1,776,875 kGal more water was saved than estimated, which is about 15% higher than the estimate. These results indicate that, overall, the reported energy savings slightly exceeded the estimated values, while estimated water savings significantly exceeded estimated values, suggesting that the projects achieved greater cost savings than originally projected. The total annual expense for the ESCOs to perform annual M&V audits and reporting was $\$$10.02million. Through this effort, $\$$647.8 million in annual cost savings was verified. The M&V results indicated that $\$$43.9 million of these verified savings reflected adjustments due to government operations A-6and maintenance impacts, which could be restored under the original operational parameters for the affected projects. These findings show the value of M&V that only costs 1.7%of the guaranteed cost savings to ensure guarantees are met.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗