To be clear, there are two new tests: (1) tests for the SARS-CoV-2 virus itself and (2) tests for antibodies against the virus. Each of these markers is now analyzed using several methods that are quickly approved by the FDA under the Emergency Use Authorization (EUA). The methods for the virus are mostly, but not all, based on PCR, and the methods for antibodies essentially all fall into the category of serological tests. PCR and other nucleic or molecular acid methods are usually performed in a section of the laboratory, depending on where the instruments for these technologies are already established. Serological tests are usually performed in another area of the laboratory. With the introduction of simple lateral flow tests, tests are also performed in point-of-care situations. All of these tests are qualitative tests, which means they have a medical decision point (threshold) to classify the result as positive or negative. In a comparative study where candidate and comparative test results are considered positive or negative, these results can be summarized as follows: Note that with a small number of samples, the confidence limits should be wide. For example, for 5 positives and 5 negatives, no false positives or false negatives, the lower limits will be about 57%; for 10 positive and 10 negative, the lower limits are about 72%; for 30, about 89%; for 40, about 91%; for 50, about 93%. All of these limitations are designed for comparisons that fit together perfectly. The smaller the number of samples tested, the lower the confidence with a single offset (false positive or false negative). [See Table A1 in EP12-A2, page 35.] This illustrates why the FDA recommends collecting at least 30 positive and 30 negative results to get unreliable estimates.
To avoid confusion, we recommend that you always use the terms opt-in consent (PPA) and opt-out consent (NPA) when describing consent to such tests. Laboratories approved by CLIA to perform medium and high complexity tests are authorized to perform manufacturer tests approved under the Emergency Use Authorization (EUA). Validation studies should continue to be conducted, and positive and negative QC samples should be analyzed at each analytical series of patient samples [1]. Although the positive and negative agreement formulas are identical to the sensitivity/specificity formulas, it is important to distinguish between them because the interpretation is different. CLSI EP12: User Protocol for Evaluation of Qualitative Test Performance protocol describes the terms Positive Percentage Agreement (PPA) and Negative Percentage Agreement (NPA). If you need to compare two binary diagnostics, you can use an agreement study to calculate these statistics. This calculator requires the user to enter 4 numbers that correspond to a (true positives), b (false positives), c (false negatives) and d (true negatives) in the contingency table. Then click on the “Calculate” button to get the summary statistics for the positive agreement (PPP), the negative agreement (NAP) and the global agreement (POA) as well as their lower and upper confidence limits of 95%. To view the example described in this lesson, click Upload Sample Data. Print the page to document your results. The FDA`s recent guidance for laboratories and manufacturers, “FDA Policy for Diagnostic Tests for Coronavirus Disease-2019 during Public Health Emergency,” states that users should use a clinical agreement study to determine performance characteristics (sensitivity/PPA, specificity/NPA). Although the terms sensitivity/specificity are widely known and used, the terms PPA/NPA are not.
In total, 100 field truth negative patients and 100 background truth positive patients were considered. In Panel A, there is no error in the classification of patients (i.e., the comparator is entirely consistent with the truth in the field). In Panel B, it is assumed that 5% of the comparator`s classifications deviate incorrectly from the truth in the field. The difference in the distribution of test results (y-axis) between the panels in this figure leads to significant underestimates of diagnostic performance, as shown in Table 1. In this scenario, truth-positive patients in the field and negative patients in the field are also likely to be misclassified by the comparator. (A) Comparator without misclassification, which perfectly represents the field truth for 100 negative patients and 100 positive patients. B) Apparent performance of the diagnostic test based on the comparator`s classification error rate. Error bars describe empirical 95% confidence intervals via medians, calculated over 100 simulation cycles.
Actual test performance is displayed when fp and FN rates are 0% each. The terms sensitivity and specificity are appropriate if there is no misclassification in the comparator (PF rate = FN rate = 0%). The terms Positive Percentage Agreement (PFA) and Negative Percentage Agreement (MPA) should be used instead of sensitivity or specificity if it is known that the comparator contains uncertainty. These calculations are not difficult, but a little chaotic. They are described in two steps, first by calculating certain quantities (Qi) from the table, and then by calculating the upper and lower confidence limits from these Qs. [Described on pages 23-25 of CLSI EP12-A2.] Nor is it possible to use these statistics to determine that one test is better than another. Recently, a British national newspaper published an article about a PCR test developed by Public Health England and the fact that it did not agree with a new commercial test in 35 of the 1144 samples (3%). Of course, for many journalists, this was proof that the PHE test was inaccurate. There is no way to know which test is good and which is wrong in any of these 35 disagreements.
.