Prospective study of PAPNET: review of 25656 Pap smears negative on manual screening and rapid rescreening

Prospective study of PAPNET: review of 25 656 Pap smears negative on manual screening and rapid rescreening J. A. HALFORD, R. G. WRIGHT AND E. J. DITCHMEN Cytology Section, Department of Anatomical Pathology, Queensland Medical Laboratory, Brisbane, Queensland, Australia Accepted for publication 2 December 1998 HALFORD J. A., WRIGHT R. G. AND DITCHMEN E. J. (1999) Cytopathology 10, 317–323 Prospective study of PAPNET: review of 25 656 Pap smears negative on manual screening and rapid rescreening In this prospective study, 27014 Pap smears were selected for PAPNET review on the request of the referring practitioner or patient. Smears that were negative on routine manual screening were submitted for rapid rescreening. Smears considered normal after these two manual screens (n 5 25656) were reviewed using the PAPNET testing system. Routine manual screening identified 1340 (4.96%) of the smears as abnormal, and a fur- ther 18 (0.07%) abnormalities were detected by rapid rescreening. PAPNET review ident- ified an additional 102 (0.4%) abnormal smears, including 10 histologically confirmed high grade lesions. The use of PAPNET testing following routine manual screening and rapid rescreening in tandem, enables cytologists to detect additional diagnostically sig- nificant abnormalities and reduce the rate of false-negative smears. Keywords: PAPNET, rapid rescreening, false-negative smears, random rescreening INTRODUCTION Despite considerable efforts to improve each of the steps in the cervical screening pathway, cervical screening has reduced but not eradicated cervical cancer. The limitations of the Pap smear, namely false-negative results, have been well documented1,2. The accuracy of the test, especially the sensitivity in detecting high grade intra- epithelial lesions, has been questioned. Considerable effort has been made to improve the detection of high grade lesions. Randomly rescreening a proportion of negative smears has been advocated as a useful method of internal quality control3. However, rapid Cytopathology 1999, 10, 317–323 © 1999 Blackwell Science Ltd Correspondence: J. A. Halford BAppSc, CT(ASC), CT(IAC), Cytology Section, Department of Anatomical Pathology, Queensland Medical Laboratory, 60 Ferry Road, West End, Brisbane, Queensland 4101, Australia. rescreening has demonstrated superior efficiency and cost effectiveness in reducing false- negative Pap smear rates4–6. Computerized rescreening devices which have been developed in recent years have demonstrated an improvement in the diagnostic accuracy of the Pap smear. The PAP- NET Testing System is a semi-automated rescreening system designed to improve sig- nificantly the detection of abnormalities7. PAPNET has demonstrated effectiveness in identifying known abnormal cases, as well as detecting abnormal cases missed on routine manual screening8–10. In a previous study, PAPNET review was shown to enhance detection of abnormal smears which had also undergone rapid rescreening11. This study was small and artificially constructed. In the present study, 25656 Pap smears considered negative on both routine manual screening and rapid rescreening were sub- mitted for review using the PAPNET Testing System. METHODS The study was carried out in a large general pathology service in Queensland, Australia, which processes over 200000 cervical smears annually. During the 10-month study period, 27014 smears were selected for PAPNET review on the request of the referring practitioner or patient. Smears selected for PAPNET review were therefore not ran- domly selected but represented routine practice during this time. Smears were reported according to the Australian (Modified Bethesda) Reporting Format (Figure 1)12. All negative smears (25674) were rapidly rescreened as described by Baker and Melcher4, using a 30-s rapid examination of the slide. Slides considered abnormal or sus- picious using the 30-s review were selected for a full manual rescreen. All abnormal slides were referred to a pathologist, and if confirmed, the report was amended. Rapid rescreen- ing has been shown to increase the detection rate of abnormal smears in our laboratory by 0.08%. The PAPNET review process is described in detail elsewhere7. A total of 25656 nega- tive slides was couriered to the PAPNET Scanning Station where computer analysis occurred using both algorithmic and neural network technology. Depending on the ‘score’ assigned by the computer, the 128 most abnormal cellular areas were photo- graphed and recorded on a digital tape. The tape together with the slides were returned to our laboratory where review of these images using the PAPNET Review Station was performed by a cytologist certified in the use of the PAPNET system. Cells were located using the X,Y co-ordinates which are displayed with the images. Slides were triaged as ‘negative’ when the images were interpreted as benign, and for ‘review’ when the images were interpreted as abnormal, suspicious, or technically unsatisfactory. All slides triaged for ‘review’ underwent full manual rescreening. Slides containing cells considered by a senior cytologist as abnormal were referred to a pathologist, and if the abnormality was confirmed the report was amended. Abnormal cases detected using rapid rescreening were also submitted for PAPNET review. RESULTS Abnormalities identified by routine manual screening, rapid rescreening and PAPNET review are summarized in Figure 1. J. A. Halford, R. G. Wright & E. J. Ditchmen © 1999 Blackwell Science Ltd, Cytopathology, 10: 317–323 318 Prospective study of PAPNET © 1999 Blackwell Science Ltd, Cytopathology, 10: 317–323 319 Figure 1. A comparison of Bethesda terminology and the Australian modification The Bethesda system The Australian modification Unsatisfactory Unsatisfactory Within normal limits Negative (1/2 Endocervical component) (1/2 Endocervical component) No abnormality No abnormality Benign/reactive changes Benign/reactive changes ASCUS LGEA Possible LGSIL Non-specific minor changes LGSIL HPV effect HPV EFFECT CIN I Inconclusive ASCUS Possible HGEA Possible HGSIL HGEA HGSIL CIN II CIN II CIN III CIN III Suggests SCC Suggests SCC SCC SCC Endocervical component present Endocervical component present Benign/reactive changes 1/2 Reactive changes AGUS LGEA Possible reactive changes Minor non-specific changes Possible adenocarcinoma in situ (Probable reactive changes) Inconclusive (Possible ACIS) HGEA Suggests adenocarcinoma Adenocarcinoma in situ Suggests adenocarcinoma Adenocarcinoma Adenocarcinoma LGSIL, Low grade squamous intraepithelial lesion; HGSIL, high grade squamous intraepithelial lesion; LGEA, low grade epithelial abnormality; HGEA, high grade epithelial abnormality; HPV, human papillomavirus; ASCUS, atypical squamous cells of undetermined significance; AGUS, atypical glandular cells of undetermined significance; CIN, cervical intraepithelial neoplasia; SCC, squamous cell carcinoma. There were 1340 smears identified as abnormal on routine manual screening, compris- ing 1186 (4.39%) cases of low grade epithelial abnormality (LGEA) (including 385 (1.43%) cervical intraepithelial neoplasia (CIN I) or human papillomavirus (HPV) infec- tion), 154 cases (0.57%) of high grade epithelial abnormality (HGEA). Of the subset of 25 674 smears reported as negative on routine screening, an extra 18 (0.071%) abnor- malities were identified using rapid rescreening. The abnormalities identified on routine screening and rapid rescreening are shown in Tables 1 and 2. Rates of detection by routine screening of LGEA and HGEA are within the performance standards specified by the National Pathology Accreditation Advisory Council (NPAAC)13. In total 102 reports were amended following PAPNET review during the study period. This represents a further increase in the detection of abnormal cases of 0.4% (Table 3). The clinical outcomes, including follow-up Pap smears, colposcopies and biopsies, for these amended cases are shown in Table 4. Eight cases were lost to follow up, including one case of ungraded dysplasia. No further follow up has been recorded on 18 of the cases since the amended results were issued. These included one case of ungraded J. A. Halford, R. G. Wright & E. J. Ditchmen © 1999 Blackwell Science Ltd, Cytopathology, 10: 317–323 320 Table 1. Abnormalities identified by routine manual screening of 27014 Pap smears Percentage of Report Number total screened LGEA 1186 4.39 HGEA 0154 0.57 Total 1340 4.96 LGEA, Low grade epithelial abnormality; HGEA, high grade epithelial abnormality. Table 2. Abnormalities identified by rapid rescreening of 25674 Pap smears, negative on routine screening Percentage of total rapid Amended result Number rescreened LGEA 16 0.063 HGEA 02 0.008 Total 18 0.071 LGEA, Low grade epithelial abnormality; HGEA, high grade epithelial abnormality. Table 3. Abnormalities identified following PAPNET review of 25656 Pap smears, negative on routine screening and rapid rescreening Percentage of Amended Number total PAPNET result amended reviews LGEA 094 0.37 HGEA 008 0.03 Total 102 0.40 LGEA, Low grade epithelial abnormality; HGEA, high grade epithelial abnormality. dysplasia from a pregnant patient. Of the 102 amended cases, 24 were confirmed as LGEA and 10 cases as HGEA. Five of these high grade cases were originally amended to LGEA on examination of the Pap smear following PAPNET review. Of the 94 cases amended to LGEA, 31 cases had a follow-up smear, colposcopy or biopsy reported as negative. All 18 cases detected by rapid rescreening were LGEA, and all were identified using the PAPNET review process. Prospective study of PAPNET © 1999 Blackwell Science Ltd, Cytopathology, 10: 317–323 321 Study 27 014 Routine screening Abnormal 1340 Normal 25 674 Rapid rescreening Abnormal 18 Normal 25 656 PAPNET review Abnormal 102 Normal 25 554 Biopsy confirmed HGSIL 10 Figure 2. Outline of the study. J. A. Halford, R. G. Wright & E. J. Ditchmen © 1999 Blackwell Science Ltd, Cytopathology, 10: 317–323 322 DISCUSSION We know of no prospective study of the cumulative benefit of rapid rescreening and PAPNET rescreening of conventional smears considered negative on manual screening. In a small artificially constructed study, PAPNET review demonstrated an enhanced detection rate for abnormalities which had also undergone rapid review11. Our present study confirms these findings. PAPNET-assisted review of slides which were negative on both routine screening and rapid rescreening identified 24 additional histologically con- firmed LGEA and 10 HGEA. Computerized rescreening of slides reported as negative has been shown to increase the detection of abnormal smears. Most of the abnormali- ties are low grade, but a significant number of high grade cases are identified10. Some abnormalities are shown to be false positive. In our study, 30 of the 94 amended to LGEA were shown to be falsely positive on follow up. Follow up for these cases com- prised mostly repeat smears, as is the recommended management for smears showing changes consistent with atypical squamous cells or HPV infection only12. Only five cases underwent colposcopy as a result of the PAPNET review, which in retrospect could be regarded as unnecessary. It should be emphasized that a diagnostic decision is not made by PAPNET. The com- puterized review process actually prompts a full manual rescreen of the slide by flagging tiles containing possible abnormal or suspicious cells. The review diagnosis is made by the cytologist (and subsequently the pathologist) following full manual review of the smear. The subjectiveness of the atypical category is well documented14 and may have been accentuated in this study by sensitivity to the PAPNET system in our laboratory. It may be argued that the overcalling of minor abnormalities is balanced by the increased detection of high grade lesions. In our study two cases of atypical squamous cells amended following PAPNET review were confirmed as HGEA on follow-up biopsies. If these patients with false-negative cytology had not undergone further investigation, there may have been the potential for progression of these lesions to invasion in the future. It is noted that the proportion of patients identified by routine manual screening with LGEA was 4.39% and with HGEA was 0.57%. This is similar to a study of the whole service, but the percentages are lower (LGEA of 5.72% and HGEA of 0.83%). An expla- nation for these findings may be that the referrals for PAPNET rescreening selected a population of women at lower risk for abnormalities and possibly reflects a bias towards a more economically affluent group. Martin15 identified this issue in an early study on cervical cancer prevention. Other authors have examined the sensitivity of PAPNET review in detecting known abnormalities7–9. In a small study in our own laboratory 19 Table 4. Clinical outcomes for abnormalities identified following PAPNET review PAPNET Negative/ No further Lost to abnormalities inflammatory LGEA HGEA follow up follow up Total LGEA 30 35 05 17 7 094 HGEA 01 05 01 1 008 Total 31 35 10 18 8 102 LGEA, Low grade epithelial abnormality; HGEA, high grade epithelial abnormality. Prospective study of PAPNET © 1999 Blackwell Science Ltd, Cytopathology, 10: 317–323 323 of 20 abnormal slides were identified by PAPNET when seeded among negative cases. In the present study the 18 cases detected using rapid rescreening were subsequently reviewed using PAPNET. All 18 cases were identified using the computer technology. Rapid rescreening has demonstrated its effectiveness as a quality control procedure in cervical screening5. The abnormal detection rate for rapid rescreening has been shown to be superior to the traditional method of randomly rescreening a selected proportion of smears6. A rapid 2-min rescreen may be more effective than the 30-s review used in this study16, but further investigation is required to evaluate the performance of PAPNET- assisted review following a 2-min rescreen. We do not know of any other computerized screening device which gives an improved benefit over routine manual screening in tandem with rapid rescreening. Automated and semi-automated devices for cervical screening are constantly changing and new develop- ments in the technology will undoubtedly demonstrate improvements in their perform- ance. Further investigation into the role of these devices is required, in particular their efficacy in primary screening. Our laboratory is currently involved in a large multicentre trial which will compare the efficacy of PAPNET as a primary screening device with conventional manual screening. The results of this and similar studies will be of benefit in the evaluation of PAPNET technology in primary screening of cervical smears. REFERENCES 1 Koss LG. The Papanicolaou test for cervical cancer detection—a triumph and a tragedy. JAMA 1989; 261: 737–43. 2 Gay JD, Donaldson LD, Goellner JR. False- negative results in cervical cytologic studies. Acta Cytol 1985; 29: 1043–6. 3 Melamed MR, Flehinger W. Reevaluation of quality assurance in the cytology laboratory (editorial). Acta Cytol 1992; 36: 461–5. 4 Hutchinson ML. Assessing the costs and bene- fits of alternative rescreening strategies. Acta Cytol 1996; 40: 4–8. 5 Baker A, Melcher DH. Rapid cervical cytology. Cytopathol 1991; 2: 299–301. 6 Johnson SJ, Hair T, Gibson L, Ridley B, Wadehra V. An assessment of partial rescreen- ing as an internal quality control method for cervical smears. Cytopathol 1995; 6: 376–87. 7 Mango LJ. Computer-assisted cervical cancer screening using neural networks. Cancer Letters 1994; 77: 155–62. 8 Rosenthal DL, Acosta D, Peters RK. Computer-assisted rescreening of clinically important false negative cervical smears using the PAPNET Testing System. Acta Cytol 1996; 40: 120–6. 9 Boon ME, Kok LP. Neural network processing can provide means to catch errors that slip through human screening of Pap smears. Diagn Cytopathol 1993; 9: 411–6. 10 Farnsworth A, Chambers FM, Goldschmidt CS. Evaluation of the PAPNET system in a general pathology service. MJA 1996; 165: 429–31. 11 Halford JA, Wright RG, Ditchmen EJ. Quality assurance in cervical cytology screening: com- parison of rapid rescreening and the PAPNET Testing System. Acta Cytol 1997; 41: 79–81. 12 National Pathology Accreditation Advisory Council. Requirements for Gynaecological (Cervical) Cytology 1997. Canberra: Australian Government Publishing Service, 1997. 13 Commonwealth Department of Health and Family Services, National Cervical Screening Program. Performance Standards for Australian Laboratories Reporting Cervical Cytology. Canberra: AGPS, 1996. 14 Davey DD, Naryshkin S, Nielsen ML, Kline TS. Atypical squamous cells of undetermined significance: interlaboratory comparison and quality assurance monitors. Diagn Cytopathol 1994; 11: 390–6. 15 Martin PL. How preventable is invasive cancer? A community study of preventable factors. Am J Obstet Gynaecol 1972; 113: 541–8. 16 Farrell DJ, Bilkhu S, Gibson LM, Cummings L, Wadehra V. Rapid rescreening of cervical smears as a method of internal quality control. For how long should we rescreen? Acta Cytol 1997; 41: 251–60.