Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “PEER Review”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Portable Common Execution Environment (PCEE) project review: Peer review

The purpose of the review was to conduct an independent, in-depth analysis of the PCEE project and to provide the results of said review. The review team was tasked with evaluating the potential contribution of the PCEE project to the improvement of the life cycle support of mission and safety critical (MASC) computing components for large, complex, non-stop, distributed systems similar to those planned for such NASA programs as the space station, lunar outpost, and manned missions to Mars. Some conclusions of the review team are as follow: The PCEE project was given high marks for its breath of vision on the overall problem with MASC software; Correlated with the sweeping vision, the Review Team is very skeptical that any research project can successfully attack such a broad range of problems; and several recommendations are made such as to identify the components of the broad solution envisioned, prioritizing them with respect to their impact and the likely ability of the PCEE or others to attack them successfully, and to rewrite its Concept Document differentiating the problem description, objectives, approach, and results so that the project vision becomes assessible to others.

Locke, C. Douglass↗

Peer Review to Peer Learning in Radiology: Where Have We Been, What Have We Learned and Where Are We Headed?

Purpose of review: To review and contrast varying methods of peer assessment driven practice quality improvement programs in radiology with an emphasis on peer review and peer learning. Recent findings: Review of the literature revealed that the current consensus is that a shift away from the original peer review system toward a peer learning process has the most beneficial effects for organizations seeking to maximally improve performance. This requires altering perceptions towards the peer review process itself, and significant time, effort, and resources. Summary: The transition to a peer learning process is a necessity to advance the field of radiology into an era of delivering near faultless quality health care.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Redefining the Practice of Peer Review Through Intelligent Automation Part 2: Data-Driven Peer Review Selection and Assignment

In conventional radiology peer review practice, a small number of exams (routinely 5% of the total volume) is randomly selected, which may significantly underestimate the true error rate within a given radiology practice. An alternative and preferable approach would be to create a data-driven model which mathematically quantifies a peer review risk score for each individual exam and uses this data to identify high risk exams and readers, and selectively target these exams for peer review. An analogous model can also be created to assist in the assignment of these peer review cases in keeping with specific priorities of the service provider. An additional option to enhance the peer review process would be to assign the peer review cases in a truly blinded fashion. In addition to eliminating traditional peer review bias, this approach has the potential to better define exam-specific standard of care, particularly when multiple readers participate in the peer review process.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Does Peer Review of Radiation Plans Affect Clinical Care? A Systematic Review of the Literature

Purpose: Peer review is a recommended component of quality assurance in radiation oncology; however, it is resource-intensive and its effect on patient care is not well understood. We conducted a systematic review of the published data to assess the reported clinical impact of peer review on radiation treatment plans. Methods and Materials: A systematic review of published English studies was performed in accordance with the PRISMA guidelines using the MEDLINE and EMBASE databases and abstracts published from major radiation oncology scientific meeting proceedings. For inclusion, the studies were required to report the effect of peer review on ≥1 element of treatment planning (eg, target volume or organ-at-risk delineation, dose prescription or dosimetry). Results: The initial search strategy identified 882 potentially eligible studies, with 11 meeting the inclusion criteria for full-text review and final analysis. Across a total of 11,491 patient cases, peer review programs led to modifications in a weighted mean of 10.8% of radiation treatment plans. Five studies differentiated between major and minor changes and reported weighted mean rates of change of 1.8% and 7.3%, respectively. The most common changes were related to target volume delineation (45.2% of changed plans), dose prescription or written directives (24.4%), and non-target volume delineation or normal tissue sparing (7.5%). Conclusions: Our findings suggest that peer review leads to changes in clinical care in approximately 1 of every 9 cases overall. This is similar to the reported rates of change in peer review studies from other oncology-related specialties, such as radiology and pathology.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Thank You to Our Peer Reviewers in 2025

Peer reviewing is the foundation of modern scholarship, with external specialists being asked to fairly check and evaluate submitted work. This difficult and often time‐consuming activity is performed voluntarily, with the understanding that one's own scholarship shall benefit down the line from a careful analysis of its assumption, results, accuracy, and yes, language, as we are now evaluating someone else's work. At Space Weather, we pride ourselves on a fair but quick review process yielding high‐quality articles with a time from submission to first decision of less than 2 months. This would not be possible without the hard work of all our reviewers. Once a year, we take the occasion to name these reviewers to thank them for their service to the journal and the community.

99 GENERAL AND MISCELLANEOUS↗

2023 IEDO Peer Review Report

On behalf of the U.S. Department of Energy (DOE) Industrial Efficiency and Decarbonization Office (IEDO), we are pleased to share the report from our 2023 peer review. Peer review provides us with invaluable insight into our office strategy and program direction from a diverse group of external thought leaders. The findings and actionable recommendations from peer review inform our decision making and help us maintain a constant state of evolution and growth.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA Product Peer Review Process

This viewgraph presentation describes NASA's product peer review process. The contents include: 1) Inspection/Peer Review at NASA; 2) Reasons for product peer reviews; 3) Different types of peer reviews; and 4) NASA requirements for peer reviews. This presentation also includes a demonstration of an actual product peer review.

Jenks, Ken↗

Water Power: 2010 Peer Review Report

This document is the peer review panel’s observations and findings, response from the Water Program to these, and supporting meeting materials including an agenda and participants list.

13 HYDRO ENERGY↗

Appendix G: Peer Review Nondisclosure Agreement

This peer review nondisclosure agreement should be signed by each reviewer prior to the program sending review materials if sensitive or proprietary information will be provided reviewers or discussed during the review, and to everyone attending a review.

99 GENERAL AND MISCELLANEOUS↗

Summary of Technical Peer Review on the Risk Assessment Framework proposed in Report INL/RPT-22-68656 for Digital Instrumentation and Control Systems

This report summarizes the peer review activities initiated by Idaho National Laboratory (INL) during fiscal year (FY) 2023 for the evaluation and improvement of the methodology developed under the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program, Risk Informed Systems Analysis (RISA) Pathway, digital instrumentation and control (DI&C) risk assessment project. In FY 2019, the RISA Pathway initiated a project to develop a risk assessment strategy for delivering a technical basis to support effective and secure DI&C technologies for digital upgrades/designs. A framework was proposed for this strategy, which aims to (1) provide a best-estimate, risk-informed capability to quantitatively and accurately estimate the risk impact of plant modernization, considering the introduction of high safety-significant safety-related (HSSSR) DI&C systems, (2) support and supplement existing risk-informed DI&C design guides by providing quantitative risk information and evidence, (3) offer a capability of design architecture evaluation of various DI&C systems, (4) assure the long-term safety and reliability of HSSSR DI&C systems, and (5) reduce uncertainty in costs and support integration of DI&C systems in the plant. The R&D efforts of this project from FY 2019 through FY 2022 were focused on methodology improvement and demonstration of the proposed framework for the risk assessment and design optimization of safety-critical DI&C systems. Collaborations with the nuclear industry have been initiated to support the reliability and risk assessment of their DI&C systems by using the proposed framework. In FY 2023, the framework has reached to a point for a technical peer review and obtain stakeholder feedback. This peer review activity includes coordination of the reviews performed by a group of industry stakeholders, documentation of the peer review suggestions, providing resolutions and responses to the peer review comments. The objective of this technical peer review is to obtain representative feedback on the proposed framework to improve the technical qualities of its methodology and readiness for deployment to the industry. Feedback may identify potential areas for improvement and further development. The Subject Matter experts were invited to review the latest project report documenting the methodology developed in the project and provide evaluations of the technical qualities of the proposed framework and relevant methods. The reviewed project report is “An Integrated Framework for Risk Assessment of High Safety-significant Safety-related Digital Instrumentation and Control Systems in Nuclear Power Plants: Methodology and Demonstration” INL/RPT-22-68656 (short as “INL/RPT-22-68656” in this report). This peer review report documents the technical questions provided for technical peer review and introduces the technical peer reviewers from the stakeholders including nuclear utilities, regulators, and universities. Comments from technical peer reviewers and the resolutions and responses to these comments are outlined. Insights and lessons learned from the technical peer review are summarized in conclusions and future work. The primary audience of this report are DI&C designers, engineers, and probabilistic risk assessment (PRA) practitioners. This includes stakeholders, such as the nuclear utilities and regulators who consider the deployment and upgrade of DI&C systems, DI&C software developers and reviewers, and cybersecurity specialists.

99 GENERAL AND MISCELLANEOUS↗

External Peer Review Team Report Underground Testing Area Subproject for Frenchman Flat, Revision 1

An external peer review was conducted to review the groundwater models used in the corrective action investigation stage of the Underground Test Area (UGTA) subproject to forecast zones of potential contamination in 1,000 years for the Frenchman Flat area. The goal of the external peer review was to provide technical evaluation of the studies and to assist in assessing the readiness of the UGTA subproject to progress to monitoring activities for further model evaluation. The external peer review team consisted of six independent technical experts with expertise in geology, hydrogeology,'''groundwater modeling, and radiochemistry. The peer review team was tasked with addressing the following questions: 1. Are the modeling approaches, assumptions, and model results for Frenchman Flat consistent with the use of modeling studies as a decision tool for resolution of environmental and regulatory requirements? 2. Do the modeling results adequately account for uncertainty in models of flow and transport in the Frenchman Flat hydrological setting? a. Are the models of sufficient scale/resolution to adequately predict contaminant transport in the Frenchman Flat setting? b. Have all key processes been included in the model? c. Are the methods used to forecast contaminant boundaries from the transport modeling studies reasonable and appropriate? d. Are the assessments of uncertainty technically sound and consistent with state-of-the-art approaches currently used in the hydrological sciences? 3. Are the datasets and modeling results adequate for a transition to Corrective Action Unit monitoring studies—the next stage in the UGTA strategy for Frenchman Flat? The peer review team is of the opinion that, with some limitations, the modeling approaches, assumptions, and model results are consistent with the use of modeling studies for resolution of environmental and regulatory requirements. The peer review team further finds that the modeling studies have accounted for uncertainty in models of flow and transport in the Frenchman Flat except for a few deficiencies described in the report. Finally, the peer review team concludes that the UGTA subproject has explored a wide range of variations in assumptions, methods, and data, and should proceed to the next stage with an emphasis on monitoring studies. The corrective action strategy, as described in the Federal Facility Agreement and Consent Order, states that the groundwater flow and transport models for each corrective action unit will consider, at a minimum, the following: • Alternative hydrostratigraphic framework models of the modeling domain. • Uncertainty in the radiological and hydrological source terms. • Alternative models of recharge. • Alternative boundary conditions and groundwater flows. • Multiple permissive sets of calibrated flow models. • Probabilistic simulations of transport using plausible sets of alternative framework and recharge models, and boundary and groundwater flows from calibrated flow models. • Ensembles of forecasts of contaminant boundaries. • Sensitivity and uncertainty analyses of model outputs. The peer review team finds that these minimum requirements have been met. While the groundwater modeling and uncertainty analyses have been quite detailed, the peer review team has identified several modeling-related issues that should be addressed in the next phase of the corrective action activities: • Evaluating and using water-level gradients from the pilot wells at the Area 5 Radioactive Waste Management Site in model calibration. • Re-evaluating the use of geochemical age-dating data to constrain model calibrations. • Developing water budgets for the alluvial and upper volcanic aquifer systems in Frenchman Flat. • Considering modeling approaches in which calculated groundwater flow directions near the water table are not predetermined by model boundary conditions and areas of recharge, all of which are very uncertain. • Evaluating local-scale variations in hydraulic conductivity on the calculated contaminant boundaries. • Evaluating the effects of non-steady-state flow conditions on calculated contaminant boundaries, including the effects of long-term declines in water levels, climatic change, and disruption of groundwater system by potential earthquake faulting along either of the two major controlling fault zones in the flow system (the Cane Spring and Rock Valley faults). • Considering the use of less-complex modeling approaches. • Evaluating the large change in water levels in the vicinity of the Frenchman Flat playa and developing a conceptual model to explain these water-level changes. • Developing a long-term groundwater level monitoring program for Frenchman Flat with regular monitoring of water levels at key monitoring wells. Despite these reservations, the peer review team strongly believes that the UGTA subproject should proceed to the next stage.

54 ENVIRONMENTAL SCIENCES↗

Peer Review Lessons Learned - DOE and Regulator Perspectives - 17140

The DOE NNSA Nevada Field Office (NFO) Underground Test Area (UGTA) Activity requires an external peer review to complete an extensive stage of site characterization and groundwater flow and contaminant transport model(s) development. The external peer review evaluates whether assumptions, methods, and conclusions derived from model(s) outputs are based on sound scientific principles and examines the scientific appropriateness of the model(s) outputs for informing the regulatory decision. The regulatory decision which is made by the Nevada Division of Environmental Protection (NDEP) is whether the model is acceptable to advance to the next UGTA strategy stage. Depending on the specific corrective action unit, the next stage is a Corrective Action Decision Document/Corrective Action Plan or Closure Stage. Three external peer reviews have taken place to date for the UGTA Activity. The first one resulted in a second phase of characterization and model development. The second external peer review resulted in advancement to the next UGTA strategy stage. While advancement to the next strategy stage was recommended by the third external peer review, a series of supplemental data collection and analyses and model simulation activities to address external peer review comments, including drilling three new wells and sampling multiple wells, occurred before moving into the next UGTA strategy stage. Twelve participants representing different aspects of the peer reviews (NDEP, NNSA/NFO, science advisors, presenters, and external peer reviewers) were interviewed to generate a combined lessons learned. Questions were asked regarding their experience with the process including panel selection, focused questions, presentations, interactions between UGTA Activity participants and reviewers, and results. The interview results were combined with the outcome of a published lessons learned report developed following the first external peer review. In general, those interviewed believed that all three external peer reviews were satisfactory and met the goals of the FFACO process. The following seven lessons learned were identified: (1) Ensure the panel is not biased and has a balance of technical and regulatory background; (2) Clearly state the review objective(s) at the start of the review; (3) Ask focused questions that support the peer review objective(s) but that do not ask the specific regulatory question; (4) Establish a peer review panel earlier in the strategy process; (5) Establish one point of contact for the entire review and conduct periodic meetings with the peer review panel to ensure focus remains on the review objective(s) and to ensure that pertinent information is provided; (6) Ensure that presentations are well coordinated, anomalous data or analysis results are addressed, and are focused on supporting the peer review objective(s); and (7) Carefully consider the panel's recommendations prior to embarking on a particular path forward. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

What's with all this peer-review stuff anyway?

The Journal of Physical Security was ostensibly started to deal with a perceived lack of peer-reviewed journals related to the field of physical security. In fact, concerns have been expressed that the field of physical security is scarcely a field at all. A typical, well-developed field might include the following: multiple peer-reviewed journals devoted to the subject, rigor and critical thinking, metrics, fundamental principles, models and theories, effective standards and guidelines, R and D conferences, professional societies, certifications, its own academic department (or at least numerous academic experts), widespread granting of degrees in the field from 4-year research universities, mechanisms for easily spotting 'snake oil' products and services, and the practice of professionals organizing to police themselves, provide quality control, and determine best practices. Physical Security seems to come up short in a number of these areas. Many of these attributes are difficult to quantify. This paper seeks to focus on one area that is quantifiable: the number of peer-reviewed journals dedicated to the field of Physical Security. In addition, I want to examine the number of overall periodicals (peer-reviewed and non-peer-reviewed) dedicated to physical security, as well as the number of papers published each year about physical security. These are potentially useful analyses because one can often infer how healthy or active a given field is by its publishing activity. For example, there are 2,754 periodicals dedicated to the (very healthy and active) field of physics. This paper concentrates on trade journal versus peer-reviewed journals. Trade journals typically focus on practice-related topics. A paper appropriate for a trade journal is usually based more on practical experience than rigorous studies or research. Models, theories, or rigorous experimental research results will usually not be included. A trade journal typically targets a specific market in an industry or trade. Such journals are often considered to be news magazines and may contain industry specific advertisements and/or job ads. A peer-reviewed journal, a.k.a 'referred journal', in contrast, contains peer-reviewed papers. A peer-reviewed paper is one that has been vetted by the peer review process. In this process, the paper is typically sent to independent experts for review and consideration. A peer-reviewed paper might cover experimental results, and/or a rigorous study, analyses, research efforts, theory, models, or one of many other scholarly endeavors.

99 GENERAL AND MISCELLANEOUS↗

2017 Water Power Project Peer Review Summary Report - Volume 1

The Water Power Technologies Office Peer Review Report summarizes the findings from a 2017 peer review held in Arlington, Virginia, with nearly 300 representatives from our national laboratories, industry, academic institutions, and other stakeholder organizations. Volume I includes sections 1–5 of the report: the executive summary, synopses of the program- and project-level evaluation results, and WPTO’s response to the 2017 Peer Review findings. This report details the results of the Water Power Peer Review Meeting, which was held in February 2017 in Arlington, VA. The purpose of the review was to evaluate WPTO-funded projects for their contribution to the mission and goals of the WPTO and DOE, assess progress against stated objectives, and appraise WPTO’s overall management and performance.

13 HYDRO ENERGY↗

Redefining the Practice of Peer Review Through Intelligent Automation—Part 3: Automated Report Analysis and Data Reconciliation

One method for addressing existing peer review limitations is the assignment of peer review cases on a completely blinded basis, in which the peer reviewer would create an independent report which can then be cross-referenced with the primary reader report of record. By leveraging existing computerized data mining techniques, one could in theory automate and objectify the process of report data extraction, classification, and analysis, while reducing time and resource requirements intrinsic to manual peer review report analysis. Once inter-report analysis has been performed, resulting inter-report discrepancies can be presented to the radiologist of record for review, along with the option to directly communicate with the peer reviewer through an electronic data reconciliation tool aimed at collaboratively resolving inter-report discrepancies and improving report accuracy. All associated report and reconciled data could in turn be recorded in a referenceable peer review database, which provides opportunity for context and user-specific education and decision support.

42 ENGINEERING↗

Water Power Technologies Office: 2022 Project Peer Review

The Water Power Technologies Office (WPTO), part of the U.S. Department of Energy’s (DOE) Office of Energy Efficiency and Renewable Energy (EERE), held a virtual peer review on July 18–29, 2022. The purpose of peer review is to evaluate DOE-funded projects for their contributions to the mission and goals of the office, progress made against stated objectives, and the office’s overall management and performance. The peer review process enables external stakeholders to provide feedback on the most impactful use of taxpayer funding and develop recommendations for the most efficient and effective ways to accelerate industry development in power technologies. This report includes the results of WPTO’s 2022 Peer Review.

13 HYDRO ENERGY↗

SU-F-T-231: Improving the Efficiency of a Radiotherapy Peer-Review System for Quality Assurance

Purpose: To improve the efficiency of a radiotherapy peer-review system using a commercially available software application for plan quality evaluation and documentation. Methods: A commercial application, FullAccess (Radialogica LLC, Version 1.4.4), was implemented in a Citrix platform for peer-review process and patient documentation. This application can display images, isodose lines, and dose-volume histograms and create plan reports for peer-review process. Dose metrics in the report can also be benchmarked for plan quality evaluation. Site-specific templates were generated based on departmental treatment planning policies and procedures for each disease site, which generally follow RTOG protocols as well as published prospective clinical trial data, including both conventional fractionation and hypo-fractionation schema. Once a plan is ready for review, the planner exports the plan to FullAccess, applies the site-specific template, and presents the report for plan review. The plan is still reviewed in the treatment planning system, as that is the legal record. Upon physician’s approval of a plan, the plan is packaged for peer review with the plan report and dose metrics are saved to the database. Results: The reports show dose metrics of PTVs and critical organs for the plans and also indicate whether or not the metrics are within tolerance. Graphical results with green, yellow, and red lights are displayed of whether planning objectives have been met. In addition, benchmarking statistics are collected to see where the current plan falls compared to all historical plans on each metric. All physicians in peer review can easily verify constraints by these reports. Conclusion: We have demonstrated the improvement in a radiotherapy peer-review system, which allows physicians to easily verify planning constraints for different disease sites and fractionation schema, allows for standardization in the clinic to ensure that departmental policies are maintained, and builds a comprehensive database for potential clinical outcome evaluation.

60 APPLIED LIFE SCIENCES↗

2010 Wind Program Peer Review Report

This report documents the evaluation of the technical, scientific, and business results of over 80 projects of the Wind Program, as well as the productivity and management effectiveness of the Wind Program itself.

17 WIND ENERGY↗