Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “QA”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Quality Assurance Plan for Federal Guidance Report 16

Federal Guidance Report No. 16 will provide risk coefficients for use in projecting the probability of cancer morbidity or mortality from exposure to environmental radionuclides. The derivation of nuclide-specific risk coefficients involves multiple steps, with each step relying on the accuracy of the previous step as well as the numerical data utilized within each step. Thus, to ensure the quality of the derived risk coefficients, it is necessary to evaluate and confirm the quality and proper implementation of input data at each step of the derivation. The Oak Ridge National Laboratory (ORNL) dosimetry research team currently has two largely independent software codes for deriving radiation dose estimates, dose coefficients, and cancer risk coefficients for internally deposited radionuclides: DCAL and QCAL. The agreement of derived values based on alternate codes is an important step in quality assurance (QA) of the risk coefficients produced in this project. DCAL is an integration of the dosimetric software and numerical databases developed at ORNL over the past 25 years for the EPA’s Office of Radiation and Indoor Air (ORIA). QCAL was developed in 2015 to serve as a substitute for DCAL while DCAL was undergoing revision and to provide QA for the updated version of DCAL. The complex methodology employed in the derivation of cancer risk coefficients presents a challenge for QA, and as such, this report describes the QA steps used by the ORNL dosimetry team in this process as implemented in DCAL and QCAL for Federal Guidance Report No. 16.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2024

Fiscal Year 2024 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued in 2024. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The SUMMAR manual was completed this fiscal year and work was started on creating manuals for SE2ANL, SE2RCT, and DASSH. Software verification work was carried out for DIF3D and REBUS in a previous program and the current fiscal year saw the completion of software verification reports for GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The goal for the next fiscal year is to complete the PERSENT software verification work and begin planning the software verification work for DASSH, SE2ANL, and SE2RCT. The code coverage reports for DIF3D and MC2-3 were completed in the previous fiscal year and the goal is to generate code coverage reports for REBUS, GAMSOR, PERSENT, and DASSH in the coming fiscal year. A considerable amount of effort was spent in the current fiscal year working on the continuous integration capability for automated regression testing in GitLab. The first version of the testing was created in the previous fiscal year and applied to DIF3D and its utility programs. That testing was extended this year to cover GAMSOR, REBUS, and PERSENT. To accomplish this, the first version of the new testing methodology had to be updated to make a single output checking methodology viable for all of the ARC software. This will result in a single document to detail the automated regression testing methodology and minor documents to detail the tolerance settings that have been applied to the output for each ARC code. The previous methodology put into place with SVN would have required a separate document for each ARC code to detail the output checking methodology and the tolerance settings for the output from each code. Because some of our industry partners are providing funds to add new capabilities to the ARC software to meet their needs, all of which must be reviewed and approved by the SQA program funded by this project, a summary of that development work is detailed in this report. Overall progress on resolving the QA gaps has been good this year with the most impactful improvement for our industry partners in capability being the creation of a threaded version of DIF3D-VARIANT that allows the DIF3D, REBUS, and GAMSOR run times to be reduced by a factor of 4-6. The most impactful QA gap that was resolved was the software verification of GAMSRC and VARPOW.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Quality assurance test of silicon photomultipliers and electronic boards for STAR event plane detector

The event plane detector (EPD), installed in the Solenoid Tracker at the Relativistic Heavy-Ion Collider located at the Brookhaven National Laboratory is a plastic scintillator-based device that measures the reaction centrality and event plane in the forward region of the relativistic heavy-ion collisions. We used silicon photomultiplier (SiPM) arrays to detect the photons produced in the scintillator via the fiber connection. Signals from the SiPM arrays were amplified by the front-end electronic (FEE) board, and sent to the analog-to-digital converter (ADC) boards for further processing via the receiver (RX) board. The full EPD system consisted of 24 super-sectors (SSs); each SS was equipped with two SiPM boards, two FEE boards and two RX boards, and they corresponded to 744 readout channels. All these boards were mass produced at the University of Science and Technology of China, with a dedicated quality assurance (QA) procedures applied to identify any problems before deployment. This article describes the details of the QA method and the related test system. As a result, the QA test results are presented along with the discussions.

EPD↗
DOE OSTI2025

Autogenerating a Domain-Specific Question-Answering Data Set from a Thermoelectric Materials Database to Enable High-Performing BERT Models

We present a method for autogenerating a large domain-specific question-answering (QA) dataset from a thermoelectric materials database. We show that a small language model, BERT, once fine-tuned on this automatically generated dataset of 99,757 QA pairs about thermoelectric materials, affords better performance in the field of thermoelectric materials compared to a BERT model fine-tuned on the generic English-language QA data set, SQuAD-v2. We further show that mixing the two data sets (ours and SQuAD-v2), which have significantly different syntactic and semantic scopes, allows the BERT model to achieve even better performance. The best-performing BERT model fine-tuned on the mixed data set outperforms the models fine-tuned on the other two data sets by scoring an exact match of 67.93% and an F1 score of 72.29% when evaluated on our test data set. This has important implications as it demonstrates the ability to realize high-performing small language models, with modest computational resources, empowered by domain-specific materials data sets which can be generated according to our method.

biological databases↗
DOE OSTI2023

Enhancing alphafold-multimer-based protein complex structure prediction with MULTICOM in CASP15

To enhance the AlphaFold-Multimer-based protein complex structure prediction, we developed a quaternary structure prediction system (MULTICOM) to improve the input fed to AlphaFold-Multimer and evaluate and refine its outputs. MULTICOM samples diverse multiple sequence alignments (MSAs) and templates for AlphaFold-Multimer to generate structural predictions by using both traditional sequence alignments and Foldseek-based structure alignments, ranks structural predictions through multiple complementary metrics, and refines the structural predictions via a Foldseek structure alignment-based refinement method. The MULTICOM system with different implementations was blindly tested in the assembly structure prediction in the 15th Critical Assessment of Techniques for Protein Structure Prediction (CASP15) in 2022 as both server and human predictors. MULTICOM_qa ranked 3 rd among 26 CASP15 server predictors and MULTICOM_human ranked 7 th among 87 CASP15 server and human predictors. The average TM-score of the first predictions submitted by MULTICOM_qa for CASP15 assembly targets is ~0.76, 5.3% higher than ~0.72 of the standard AlphaFold-Multimer. The average TM-score of the best of top 5 predictions submitted by MULTICOM_qa is ~0.80, about 8% higher than ~0.74 of the standard AlphaFold-Multimer. Moreover, the Foldseek Structure Alignment-based Multimer structure Generation (FSAMG) method outperforms the widely used sequence alignment-based multimer structure generation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

A novel path to runaway electron mitigation via deuterium injection and current-driven MHD instability

Relativistic electron (RE) beams at high current density (low safety factor, qa) yet very low free-electron density accessed with D2 secondary injection in the DIII-D and JET tokamak are found to exhibit large-scale MHD instabilities that benignly terminate the RE beam. In JET, this technique has enabled termination of MA-level RE currents without measurable first-wall heating. This scenario thus offers an unexpected alternate pathway to achieve RE mitigation without collisional dissipation. Benign termination is explained by two synergistic effects. First, during the MHD-driven RE loss events both experiment and MHD orbit-loss modeling supports a significant increase in the wetted area of the RE loss. Second, as previously identified at JET and DIII-D, the fast kink loss timescale precludes RE beam regeneration and the resulting dangerous conversion of magnetic to RE kinetic energy. During the termination, the RE kinetic energy is lost to the wall, but the current fully transfers to the cold bulk thus enabling benign Ohmic dissipation of the magnetic energy on longer timescales via a conventional current quench. Hydrogenic (D2) secondary injection is found to be the only injected species that enables access to the benign termination. D2 injection: 1) facilitates access to low qa in existing devices (via reduced collisionality & resistivity), 2) minimizes the RE avalanche by ‘purging’ the high-Z atoms from the RE beam, 3) drives recombination of the background plasma, reducing the density and Alfven time, thus accelerating the MHD growth. Furthermore, this phenomenon is found to be accessible when crossing the low qa stability boundary with rising current, falling toroidal field, or contracting minor radius - the latter being the expected scenario for vertically unstable RE beams in ITER. While unexpected, this path scales favorably to fusion-grade tokamaks and offers a novel RE mitigation scenario in principle accessible with the day-one disruption mitigation system (DMS) of ITER.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Establishing the Quality Assurance programme for the strip sensor production of the ATLAS tracker upgrade including irradiation with neutrons, photons and protons to HL-LHC fluences

The successful pre-production delivery of strip sensors for the new Inner Tracker (ITk) for the upgraded ATLAS detector at the High Luminosity LHC (HL-LHC) at CERN was completed and based on their performance full production has commenced. The overall delivery period is anticipated to last 4 years to complete the approximately 22000 sensors required for the ITk. For Quality Assurance (QA), a number of test structures designed by the collaboration, along with a large area diode and miniature version of the main sensor, are produced in every wafer by the foundry Hamamatsu Photonics K.K (HPK). As well as Quality Control (QC) checks on every main sensor, samples of the QA pieces from each delivery batch are tested both before and after irradiation with results after exposure to neutrons, gammas or protons to fluences and doses corresponding to those anticipated after operation at the HL-LHC to roughly 1.5 times the ultimate integrated luminosity of 4000 fb -1 . In this paper the procedures are presented and the studies carried out to establish that the seven ITk QA Strip Sensor irradiation and test sites meet all the requirements to support this very extensive programme throughout the strip sensor production phase for the ITk project.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Diabatic quantum annealing for the frustrated ring model

Abstract Quantum annealing (QA) is a continuous-time heuristic quantum algorithm for solving or approximately solving classical optimization problems. The algorithm uses a schedule to interpolate between a driver Hamiltonian with an easy-to-prepare ground state and a problem Hamiltonian whose ground state encodes solutions to an optimization problem. The standard implementation relies on the evolution being adiabatic: keeping the system in the instantaneous ground state with high probability and requiring a time scale inversely related to the minimum energy gap between the instantaneous ground and excited states. However, adiabatic evolution can lead to evolution times that scale exponentially with the system size, even for computationally simple problems. Here, we study whether non-adiabatic evolutions with optimized annealing schedules can bypass this exponential slowdown for one such class of problems called the frustrated ring model. For sufficiently optimized annealing schedules and system sizes of up to 39 qubits, we provide numerical evidence that we can avoid the exponential slowdown. Our work highlights the potential of highly-controllable QA to circumvent bottlenecks associated with the standard implementation of QA.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Anomaly Detection in Seismic Data with Deep Learning: Application for Instrument Failure Detection and Forecasting

Seismic data quality assessment (QA) is the first and one of the most important steps before conducting any further data analysis. Traditional methods involve checking various metrics, such as spike detection and power spectral density, by setting strict thresholds or comparing data against synthetic benchmarks. However, these approaches often rely on pre-existing knowledge and assumptions about data anomalies, leading to potential misclassification of unusual cases. Here, in this study, we propose a deep autoencoder model, an unsupervised learning approach that evaluates data quality without making assumptions about normal and anomalous data, which can be used to identify deviations in recorded data that may indicate nascent instrument failure. We test the model with the U.S. International Monitoring System (IMS) seismic stations and demonstrate the capability of detecting anomalies on a monthly scale. This could prompt station operators to examine potential problems early, allowing sufficient time for instrument maintenance to prevent data outages. In addition, we use a new manually selected testing dataset to compare our model performance against two supervised machine learning (ML) approaches and a standard QA package, as baseline models. When applied to the dataset containing known data anomalies, performance of the supervised and unsupervised ML approaches is similar, with an accuracy of 88.1% for our model compared to ∼90% for the supervised ML approach and 78.2% for the standard QA package. Our model outperforms the baseline models when applied to new stations, where new types of data anomalies can be station-specific and not included in the training dataset. Finally, we show model transferability by training the model with data from the Global Seismograph Network only and applying it to the IMS network data. The results suggest that our model is generalizable and can be applied to new stations with good accuracy.

Lin, Jiun-Ting [Lawrence Livermore National Labora↗
DOE OSTI2020

PV Fleet Performance Data Initiative (March 2020 Methodology Report)

Data for thousands of systems by multiple sources were transmitted to NREL for evaluation. Initial automated data quality assurance (QA) checks identified hundreds of systems (QA Tier 1) with high-quality meteorological and AC production data and hundreds of QA Tier 2 systems with adequate data quality. A standard RdTools analysis was conducted for these systems, evaluating performance loss on an annualized basis. To date with the systems evaluated so far, we find a median performance loss rate (aka Rd or degradation rates) for the entire fleet of -0.72%/year based on individual inverter data, which is in line with historical degradation rates previously published for modules and systems (-0.5% to -0.9% / yr, Jordan et al. 2016). A fleet-scale distribution based on revenue-grade meter data rather than inverter AC (alternating current) data yields a slightly different result. We are working to understand the difference between this result and the inverter-based result of -0.72%/year.

14 SOLAR ENERGY↗
DOE OSTI2022

Verification and Validation of a Modified Numerical Algorithm for Simulation of Transient Unconfined Groundwater Flow

This report extends verification and validation testing of the NUFT package of codes (Nitao 2000a,b) to the US1P module of code designed for simulation of single-phase mixed or “coupled” saturated and unsaturated (or “variably saturated”) groundwater flow. Importantly, the verification and validation testing of US1P in this report includes performance evaluation of a modified numerical algorithm that was not included in prior quality assurance (QA) of the NUFT package by Carle et al. (2014) for Underground Test Area (UGTA) project activities, which are directed at assessment of radionuclide contamination in groundwater sourced from underground nuclear test locations at the Nevada National Security Site (NNSS). An immediate purpose of this report is to provide QA for ongoing large-scale, three-dimensional (3-D) groundwater flow modeling of transient water levels associated with longterm water supply pumping and underground nuclear testing at the NNSS (Jackson and Fenelon, 2018; Jackson et al., 2021). For UGTA activities, the QA refers to the standards of ASTM (1996) for verification and validation testing of groundwater modeling codes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Underground Test Area: Calendar Year 2022 Quality Assurance Report, Nevada National Security Site, Nevada

This report is required by the Underground Test Area (UGTA) Activity QAP and identifies the UGTA QA activities for CY 2022. The QA activities included conducting assessments for UGTA Activity QAP compliance, identifying findings and completing corrective actions, evaluating laboratory performance, reviewing technical work, and publishing documents. DRI; LANL; and MSTS did not conduct QA activities for the UGTA Activity in 2022.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued through 2024 and 2025. The continuous integration testing was extended to RCT, DASSH, and SE2ANL. Minor changes were required to the original continuous integration methodology to make this happen. When full confidence in the methodology is complete, a report will be created to detail the automated regression testing methodology and minor reports will be created to detail the tolerance settings that have been applied to the output for each ARC code. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The DASSH, SE2ANL, and SE2RCT manuals were completed this fiscal year. A review of the SE2ANL software identified that it is unrealistic to include updated correlations or different geometry models and it was scheduled for deprecation in favor of DASSH. The SE2ANL manual is essential for SE2RCT as they are similar but quite different in purpose. The only piece of software missing a manual consistent with the source code is NUBOW-3D which is a focus of the coming year. The code coverage report for DIF3D was updated and code coverage reports were created for REBUS, RCT, PERSENT, GAMSRC, and DASSH. Minor coverage issues were identified for all of these pieces of software which did not prevent the work done to transition them to the OneAPI compiler. Because SE2ANL was scheduled for deprecation, it was not transitioned, but it was successfully tested with the OneAPI compiler. This leaves SE2RCT and NUBOW-3D as the only pieces of software not transitioned to OneAPI and further work is required to get SE2RCT to work properly. The SE2RCT software transition will begin early next year while the NUBOW-3D software requires a manual before it can begin. Software verification work has been completed for DIF3D, REBUS, GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The PERSENT software verification work was completed this year which was somewhat delayed because of unexpected bugs in the software. The PERSENT manual was updated to detail some of the issues and discuss the bowing reactivity worth feature added in the previous fiscal year. The RCT, DASSH, SE2RCT, and NUBOW-3D software are the only maintained pieces of software without verification reports. The software verification work for DASSH will be a focus in the upcoming fiscal year and it is hoped that some of the test cases created can serve as verification tests for SE2RCT. The NUBOW-3D work will begin when the manual and requirements report are completed. Only minor industry partner software development funds were provided this year. The DASSH software was updated to handle general axial geometry for each assembly and the NUBOW-3D software was updated to incorporate a new input format and better output. Overall progress on resolving the QA gaps has been good this year.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Fiscal Year 2025 Software Quality Assurance Activities for the ARC Software

The continued goal of the ARC SQA project in the Advanced Reactor Technologies program of DOE is to resolve the QA gaps for the ARC software that limit, or prevent, commercialization of the software for industry users. This project started in earnest in fiscal year 2023 which saw the entire code system moved from a SVN repository to a GitLab repository and an associated software quality assurance plan (SQAP) developed and ratified. Most of the QA gaps in the ARC software were identified in collaboration with industry partners and work begin in fiscal year 2023 and continued through 2024 and 2025. The continuous integration testing was extended to RCT, DASSH, and SE2ANL. Minor changes were required to the original continuous integration methodology to make this happen. When full confidence in the methodology is complete, a report will be created to detail the automated regression testing methodology and minor reports will be created to detail the tolerance settings that have been applied to the output for each ARC code. The primary documentation that is missing includes user manuals, user guides, software verification reports, and code coverage assessments. The DASSH, SE2ANL, and SE2RCT manuals were completed this fiscal year. A review of the SE2ANL software identified that it is unrealistic to include updated correlations or different geometry models and it was scheduled for deprecation in favor of DASSH. The SE2ANL manual is essential for SE2RCT as they are similar but quite different in purpose. The only piece of software missing a manual consistent with the source code is NUBOW-3D which is a focus of the coming year. The code coverage report for DIF3D was updated and code coverage reports were created for REBUS, RCT, PERSENT, GAMSRC, and DASSH. Minor coverage issues were identified for all of these pieces of software which did not prevent the work done to transition them to the OneAPI compiler. Because SE2ANL was scheduled for deprecation, it was not transitioned, but it was successfully tested with the OneAPI compiler. This leaves SE2RCT and NUBOW-3D as the only pieces of software not transitioned to OneAPI and further work is required to get SE2RCT to work properly. The SE2RCT software transition will begin early next year while the NUBOW-3D software requires a manual before it can begin. Software verification work has been completed for DIF3D, REBUS, GAMSOR, GAMSRC, VARPOW, EvaluateFlux, and SUMMAR. The PERSENT software verification work was completed this year which was somewhat delayed because of unexpected bugs in the software. The PERSENT manual was updated to detail some of the issues and discuss the bowing reactivity worth feature added in the previous fiscal year. The RCT, DASSH, SE2RCT, and NUBOW-3D software are the only maintained pieces of software without verification reports. The software verification work for DASSH will be a focus in the upcoming fiscal year and it is hoped that some of the test cases created can serve as verification tests for SE2RCT. The NUBOW-3D work will begin when the manual and requirements report are completed. Only minor industry partner software development funds were provided this year. The DASSH software was updated to handle general axial geometry for each assembly and the NUBOW-3D software was updated to incorporate a new input format and better output. Overall progress on resolving the QA gaps has been good this year.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Waste Glass Property Database and Data Qualification Plan

The U.S. Department of Energy vitrification facilities currently employ glass property-composition models to ensure processability of waste streams and acceptability of the final glass products. The efficiency (e.g., process flexibility, reduced down time, and reduced cost) of waste vitrification is directly tied to the size of the waste processing envelope. To increase the size of the processing envelope, a larger database was developed. The database will improve the prediction accuracy of glass properties, leading to higher waste loadings, higher waste throughput, broader process flexibility, and reduced mission life. Some of the data in this expanded database has insufficient quality assurance (QA) rigor for use in nuclear facility operation and glass qualification. A plan was developed to qualify high-value data under the appropriate QA rigor for use at the Hanford Waste Treatment and Immobilization Plant’s High-Level Waste (HLW) Facility. This qualification plan first compares the data coverage over the composition region of interest to Hanford HLW by property. Those studies that significantly improve the data coverage in the Hanford HLW glass composition region were prioritized for qualification activities. Qualification methods including QA equivalence, peer review, data corroboration, and confirmatory testing have been assigned to each high-priority study.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

PVAnalytics [SWR-20-33]

PVAnalytics inputs long term (multiple years) photovoltaic (PV) outdoor sensor data and completes autonomous quality assurance (QA) analysis on the data. Time series data from individual sensors such as electric meters, inverter outputs, irradiance, and temperature sensors are compared to physically expected outputs of these sensors under clear sky conditions based on the latitude, longitude, and orientation of the PV system. PVAnalytics outputs as QA pass or fail for each sensor, reasons determined for failure, time series flags identifying QA thresholds have been exceeded, flags for sunny days, the time zone found for the given sensor, sensor drift, time periods for passing data, and results of orientation checks.

Muller, Matthew↗
DOE OSTI2022

PVAnalytics: A Python Package for Automated Processing of Solar Time Series Data

Multiple publicly available software packages exist that analyze solar time series data, including RdTools and Solar Data Tools, among others. Several of these packages contain their own unique quality assurance (QA) and feature recognition algorithms. The python PVAnalytics package was developed to offer an internally consistent source for these analysis tools, making it easier for the end user to deploy these routines on his or her solar data. The PVAnalytics package currently contains routines for outlier detection, inverter clipping detection, irradiance and temperature checks, orientation checks, and data shift detection, among other functions. These functions have been aggregated from various sources including Solar Forecast Arbiter, RdTools, and the QA process developed by NREL's PV Fleets Initiative. We are continuously adding new functionality to the package, including documentation, examples and algorithms. By bundling QA functionality into a single software package, we hope to make PVAnalytics a comprehensive software library to support analysis of solar metadata and time series data.

data cleaning↗
DOE OSTI2023

Combining pairwise structural similarity and deep learning interface contact prediction to estimate protein complex model accuracy in CASP15

Abstract Estimating the accuracy of quaternary structural models of protein complexes and assemblies (EMA) is important for predicting quaternary structures and applying them to studying protein function and interaction. The pairwise similarity between structural models is proven useful for estimating the quality of protein tertiary structural models, but it has been rarely applied to predicting the quality of quaternary structural models. Moreover, the pairwise similarity approach often fails when many structural models are of low quality and similar to each other. To address the gap, we developed a hybrid method (MULTICOM_qa) combining a pairwise similarity score (PSS) and an interface contact probability score (ICPS) based on the deep learning inter‐chain contact prediction for estimating protein complex model accuracy. It blindly participated in the 15th Critical Assessment of Techniques for Protein Structure Prediction (CASP15) in 2022 and performed very well in estimating the global structure accuracy of assembly models. The average per‐target correlation coefficient between the model quality scores predicted by MULTICOM_qa and the true quality scores of the models of CASP15 assembly targets is 0.66. The average per‐target ranking loss in using the predicted quality scores to rank the models is 0.14. It was able to select good models for most targets. Moreover, several key factors (i.e., target difficulty, model sampling difficulty, skewness of model quality, and similarity between good/bad models) for EMA are identified and analyzed. The results demonstrate that combining the multi‐model method (PSS) with the complementary single‐model method (ICPS) is a promising approach to EMA.

59 BASIC BIOLOGICAL SCIENCES↗