Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader

Ashkan M. Abbey, Cagri G. Besirli, David C. Musch, Chris A. Andrews, Antonio Capone, Kimberly A. Drenser, David K. Wallace, Susan Ostmo, Michael Chiang, Paul P. Lee, Michael T. Trese

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Purpose To determine if (1) tortuosity assessment by a computer program (ROPtool, developed at the University of North Carolina, Chapel Hill, and Duke University, and licensed by FocusROP) that traces retinal blood vessels and (2) assessment by a lay reader are comparable with assessment by a panel of 3 retinopathy of prematurity (ROP) experts for remote clinical grading of vascular abnormalities such as plus disease. Design Validity and reliability analysis of diagnostic tools. Participants Three hundred thirty-five fundus images of prematurely born infants. Methods Three hundred thirty-five fundus images of prematurely born infants were obtained by neonatal intensive care unit nurses. A panel of 3 ROP experts graded 84 images showing vascular dilatation, tortuosity, or both and 251 images showing no evidence of vascular abnormalities. These images were sent electronically to an experienced lay reader who independently graded them for vascular abnormalities. The images also were analyzed using the ROPtool, which assigns a numerical value to the level of vascular abnormality and tortuosity present in each of 4 quadrants or sectors. The ROPtool measurements of vascular abnormalities were graded and compared with expert panel grades with a receiver operating characteristic (ROC) curve. Grades between human readers were cross-tabulated. The area under the ROC curve was calculated for the ROPtool, and sensitivity and specificity were computed for the lay reader. Main Outcome Measures Measurements of vascular abnormalities by ROPtool and grading of vascular abnormalities by 3 ROP experts and 1 experienced lay reader. Results The ROC curve for ROPtool's tortuosity assessment had an area under the ROC curve of 0.917. Using a threshold value of 4.97 for the second most tortuous quadrant, ROPtool's sensitivity was 91% and its specificity was 82%. Lay reader sensitivity and specificity were 99% and 73%, respectively, and had high reliability (κ, 0.87) in repeated measurements. Conclusions ROPtool had very good accuracy for detection of vascular abnormalities suggestive of plus disease when compared with expert physician graders. The lay reader's results showed excellent sensitivity and good specificity when compared with those of the expert graders. These options for remote reading of images to detect vascular abnormalities deserve consideration in the quest to use telemedicine with remote reading for efficient delivery of high-quality care and to detect infants requiring bedside examination.

Original languageEnglish (US)
Pages (from-to)385-390
Number of pages6
JournalOphthalmology
Volume123
Issue number2
DOIs
StatePublished - Jan 1 2016
Externally publishedYes

Fingerprint

Retinopathy of Prematurity
Blood Vessels
ROC Curve
Sensitivity and Specificity
Reading
Retinal Vessels
Telemedicine
Quality of Health Care
Neonatal Intensive Care Units
Reproducibility of Results
Dilatation
Software
Nurses
Outcome Assessment (Health Care)
Physicians

ASJC Scopus subject areas

  • Ophthalmology

Cite this

Abbey, A. M., Besirli, C. G., Musch, D. C., Andrews, C. A., Capone, A., Drenser, K. A., ... Trese, M. T. (2016). Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader. Ophthalmology, 123(2), 385-390. https://doi.org/10.1016/j.ophtha.2015.09.048

Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader. / Abbey, Ashkan M.; Besirli, Cagri G.; Musch, David C.; Andrews, Chris A.; Capone, Antonio; Drenser, Kimberly A.; Wallace, David K.; Ostmo, Susan; Chiang, Michael; Lee, Paul P.; Trese, Michael T.

In: Ophthalmology, Vol. 123, No. 2, 01.01.2016, p. 385-390.

Research output: Contribution to journalArticle

Abbey, AM, Besirli, CG, Musch, DC, Andrews, CA, Capone, A, Drenser, KA, Wallace, DK, Ostmo, S, Chiang, M, Lee, PP & Trese, MT 2016, 'Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader', Ophthalmology, vol. 123, no. 2, pp. 385-390. https://doi.org/10.1016/j.ophtha.2015.09.048
Abbey AM, Besirli CG, Musch DC, Andrews CA, Capone A, Drenser KA et al. Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader. Ophthalmology. 2016 Jan 1;123(2):385-390. https://doi.org/10.1016/j.ophtha.2015.09.048
Abbey, Ashkan M. ; Besirli, Cagri G. ; Musch, David C. ; Andrews, Chris A. ; Capone, Antonio ; Drenser, Kimberly A. ; Wallace, David K. ; Ostmo, Susan ; Chiang, Michael ; Lee, Paul P. ; Trese, Michael T. / Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader. In: Ophthalmology. 2016 ; Vol. 123, No. 2. pp. 385-390.
@article{d3cd06d44baa4ab7967d85ee4e852867,
title = "Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader",
abstract = "Purpose To determine if (1) tortuosity assessment by a computer program (ROPtool, developed at the University of North Carolina, Chapel Hill, and Duke University, and licensed by FocusROP) that traces retinal blood vessels and (2) assessment by a lay reader are comparable with assessment by a panel of 3 retinopathy of prematurity (ROP) experts for remote clinical grading of vascular abnormalities such as plus disease. Design Validity and reliability analysis of diagnostic tools. Participants Three hundred thirty-five fundus images of prematurely born infants. Methods Three hundred thirty-five fundus images of prematurely born infants were obtained by neonatal intensive care unit nurses. A panel of 3 ROP experts graded 84 images showing vascular dilatation, tortuosity, or both and 251 images showing no evidence of vascular abnormalities. These images were sent electronically to an experienced lay reader who independently graded them for vascular abnormalities. The images also were analyzed using the ROPtool, which assigns a numerical value to the level of vascular abnormality and tortuosity present in each of 4 quadrants or sectors. The ROPtool measurements of vascular abnormalities were graded and compared with expert panel grades with a receiver operating characteristic (ROC) curve. Grades between human readers were cross-tabulated. The area under the ROC curve was calculated for the ROPtool, and sensitivity and specificity were computed for the lay reader. Main Outcome Measures Measurements of vascular abnormalities by ROPtool and grading of vascular abnormalities by 3 ROP experts and 1 experienced lay reader. Results The ROC curve for ROPtool's tortuosity assessment had an area under the ROC curve of 0.917. Using a threshold value of 4.97 for the second most tortuous quadrant, ROPtool's sensitivity was 91{\%} and its specificity was 82{\%}. Lay reader sensitivity and specificity were 99{\%} and 73{\%}, respectively, and had high reliability (κ, 0.87) in repeated measurements. Conclusions ROPtool had very good accuracy for detection of vascular abnormalities suggestive of plus disease when compared with expert physician graders. The lay reader's results showed excellent sensitivity and good specificity when compared with those of the expert graders. These options for remote reading of images to detect vascular abnormalities deserve consideration in the quest to use telemedicine with remote reading for efficient delivery of high-quality care and to detect infants requiring bedside examination.",
author = "Abbey, {Ashkan M.} and Besirli, {Cagri G.} and Musch, {David C.} and Andrews, {Chris A.} and Antonio Capone and Drenser, {Kimberly A.} and Wallace, {David K.} and Susan Ostmo and Michael Chiang and Lee, {Paul P.} and Trese, {Michael T.}",
year = "2016",
month = "1",
day = "1",
doi = "10.1016/j.ophtha.2015.09.048",
language = "English (US)",
volume = "123",
pages = "385--390",
journal = "Ophthalmology",
issn = "0161-6420",
publisher = "Elsevier Inc.",
number = "2",

}

TY - JOUR

T1 - Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader

AU - Abbey, Ashkan M.

AU - Besirli, Cagri G.

AU - Musch, David C.

AU - Andrews, Chris A.

AU - Capone, Antonio

AU - Drenser, Kimberly A.

AU - Wallace, David K.

AU - Ostmo, Susan

AU - Chiang, Michael

AU - Lee, Paul P.

AU - Trese, Michael T.

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Purpose To determine if (1) tortuosity assessment by a computer program (ROPtool, developed at the University of North Carolina, Chapel Hill, and Duke University, and licensed by FocusROP) that traces retinal blood vessels and (2) assessment by a lay reader are comparable with assessment by a panel of 3 retinopathy of prematurity (ROP) experts for remote clinical grading of vascular abnormalities such as plus disease. Design Validity and reliability analysis of diagnostic tools. Participants Three hundred thirty-five fundus images of prematurely born infants. Methods Three hundred thirty-five fundus images of prematurely born infants were obtained by neonatal intensive care unit nurses. A panel of 3 ROP experts graded 84 images showing vascular dilatation, tortuosity, or both and 251 images showing no evidence of vascular abnormalities. These images were sent electronically to an experienced lay reader who independently graded them for vascular abnormalities. The images also were analyzed using the ROPtool, which assigns a numerical value to the level of vascular abnormality and tortuosity present in each of 4 quadrants or sectors. The ROPtool measurements of vascular abnormalities were graded and compared with expert panel grades with a receiver operating characteristic (ROC) curve. Grades between human readers were cross-tabulated. The area under the ROC curve was calculated for the ROPtool, and sensitivity and specificity were computed for the lay reader. Main Outcome Measures Measurements of vascular abnormalities by ROPtool and grading of vascular abnormalities by 3 ROP experts and 1 experienced lay reader. Results The ROC curve for ROPtool's tortuosity assessment had an area under the ROC curve of 0.917. Using a threshold value of 4.97 for the second most tortuous quadrant, ROPtool's sensitivity was 91% and its specificity was 82%. Lay reader sensitivity and specificity were 99% and 73%, respectively, and had high reliability (κ, 0.87) in repeated measurements. Conclusions ROPtool had very good accuracy for detection of vascular abnormalities suggestive of plus disease when compared with expert physician graders. The lay reader's results showed excellent sensitivity and good specificity when compared with those of the expert graders. These options for remote reading of images to detect vascular abnormalities deserve consideration in the quest to use telemedicine with remote reading for efficient delivery of high-quality care and to detect infants requiring bedside examination.

AB - Purpose To determine if (1) tortuosity assessment by a computer program (ROPtool, developed at the University of North Carolina, Chapel Hill, and Duke University, and licensed by FocusROP) that traces retinal blood vessels and (2) assessment by a lay reader are comparable with assessment by a panel of 3 retinopathy of prematurity (ROP) experts for remote clinical grading of vascular abnormalities such as plus disease. Design Validity and reliability analysis of diagnostic tools. Participants Three hundred thirty-five fundus images of prematurely born infants. Methods Three hundred thirty-five fundus images of prematurely born infants were obtained by neonatal intensive care unit nurses. A panel of 3 ROP experts graded 84 images showing vascular dilatation, tortuosity, or both and 251 images showing no evidence of vascular abnormalities. These images were sent electronically to an experienced lay reader who independently graded them for vascular abnormalities. The images also were analyzed using the ROPtool, which assigns a numerical value to the level of vascular abnormality and tortuosity present in each of 4 quadrants or sectors. The ROPtool measurements of vascular abnormalities were graded and compared with expert panel grades with a receiver operating characteristic (ROC) curve. Grades between human readers were cross-tabulated. The area under the ROC curve was calculated for the ROPtool, and sensitivity and specificity were computed for the lay reader. Main Outcome Measures Measurements of vascular abnormalities by ROPtool and grading of vascular abnormalities by 3 ROP experts and 1 experienced lay reader. Results The ROC curve for ROPtool's tortuosity assessment had an area under the ROC curve of 0.917. Using a threshold value of 4.97 for the second most tortuous quadrant, ROPtool's sensitivity was 91% and its specificity was 82%. Lay reader sensitivity and specificity were 99% and 73%, respectively, and had high reliability (κ, 0.87) in repeated measurements. Conclusions ROPtool had very good accuracy for detection of vascular abnormalities suggestive of plus disease when compared with expert physician graders. The lay reader's results showed excellent sensitivity and good specificity when compared with those of the expert graders. These options for remote reading of images to detect vascular abnormalities deserve consideration in the quest to use telemedicine with remote reading for efficient delivery of high-quality care and to detect infants requiring bedside examination.

UR - http://www.scopus.com/inward/record.url?scp=84949636811&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84949636811&partnerID=8YFLogxK

U2 - 10.1016/j.ophtha.2015.09.048

DO - 10.1016/j.ophtha.2015.09.048

M3 - Article

VL - 123

SP - 385

EP - 390

JO - Ophthalmology

JF - Ophthalmology

SN - 0161-6420

IS - 2

ER -