Prospective multicenter assessment of interobserver agreement for radiologist interpretation of multidetector computerized tomographic angiography for pulmonary embolism

D. M. Courtney, C. Miller, H. Smithline, N. Klekowski, M. Hogg, Jeffrey Kline

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Background: Emergency physicians rely on the interpretation of radiologists to diagnose and exclude pulmonary embolism (PE) on the basis of computerized tomographic pulmonary angiography (CTPA). Few data exist regarding the interobserver reliability of this endpoint. Objective: To quantify the degree of agreement in CTPA interpretation between four academic hospitals and an independent reference reading (IRR) laboratory. Methods: Hospitalized and emergency department patients who had one predefined symptom and sign of PE and underwent 64-slice CTPA were enrolled from four academic hospitals. CTPA results as interpreted by board-certified radiologists from the hospitals were compared against those from the IRR laboratory. CTPAs were read as indeterminate, PE- or PE+, and percentage obstruction was computed by the IRR laboratory, using a published method. Agreement was calculated with weighted Cohen's kappa. Results: We enrolled 492 subjects (63% female, age 54 ± 1 years, and 16.7% PE+ at the site hospitals). Overall agreement was 429/492 (87.2%; 95% confidence interval 83.9-90.0). We observed 13 cases (2.6%) of complete discordance, where one reading was PE+ and the other reading was PE-. Weighted agreement was 92.3%, with kappa = 0.75. The median percentage obstruction for all patients was 9% (25th-75th percentile interquartile range: 5% to - 30%). For CTPAs interpreted at the site hospitals as PE- or indeterminate but read as PE+ by the IRR laboratory, the median of percentage obstruction was 6% (4-7%). Conclusion: We found in this sample a good level of agreement, with a weighted kappa of 0.75, but with 2.6% of patients having total discordance. Overall, a large proportion of clots were distal or minimally occlusive clots.

Original languageEnglish (US)
Pages (from-to)533-539
Number of pages7
JournalJournal of Thrombosis and Haemostasis
Volume8
Issue number3
DOIs
StatePublished - Mar 2010
Externally publishedYes

Fingerprint

Pulmonary Embolism
Angiography
Reading
Lung
Radiologists
Signs and Symptoms
Hospital Emergency Service
Emergencies
Confidence Intervals
Physicians

Keywords

  • Computed tomography
  • Interobserver agreement
  • Pulmonary embolism
  • Reliability

ASJC Scopus subject areas

  • Hematology

Cite this

Prospective multicenter assessment of interobserver agreement for radiologist interpretation of multidetector computerized tomographic angiography for pulmonary embolism. / Courtney, D. M.; Miller, C.; Smithline, H.; Klekowski, N.; Hogg, M.; Kline, Jeffrey.

In: Journal of Thrombosis and Haemostasis, Vol. 8, No. 3, 03.2010, p. 533-539.

Research output: Contribution to journalArticle

@article{357a5922ba81403c9a6953bd31d3108f,
title = "Prospective multicenter assessment of interobserver agreement for radiologist interpretation of multidetector computerized tomographic angiography for pulmonary embolism",
abstract = "Background: Emergency physicians rely on the interpretation of radiologists to diagnose and exclude pulmonary embolism (PE) on the basis of computerized tomographic pulmonary angiography (CTPA). Few data exist regarding the interobserver reliability of this endpoint. Objective: To quantify the degree of agreement in CTPA interpretation between four academic hospitals and an independent reference reading (IRR) laboratory. Methods: Hospitalized and emergency department patients who had one predefined symptom and sign of PE and underwent 64-slice CTPA were enrolled from four academic hospitals. CTPA results as interpreted by board-certified radiologists from the hospitals were compared against those from the IRR laboratory. CTPAs were read as indeterminate, PE- or PE+, and percentage obstruction was computed by the IRR laboratory, using a published method. Agreement was calculated with weighted Cohen's kappa. Results: We enrolled 492 subjects (63{\%} female, age 54 ± 1 years, and 16.7{\%} PE+ at the site hospitals). Overall agreement was 429/492 (87.2{\%}; 95{\%} confidence interval 83.9-90.0). We observed 13 cases (2.6{\%}) of complete discordance, where one reading was PE+ and the other reading was PE-. Weighted agreement was 92.3{\%}, with kappa = 0.75. The median percentage obstruction for all patients was 9{\%} (25th-75th percentile interquartile range: 5{\%} to - 30{\%}). For CTPAs interpreted at the site hospitals as PE- or indeterminate but read as PE+ by the IRR laboratory, the median of percentage obstruction was 6{\%} (4-7{\%}). Conclusion: We found in this sample a good level of agreement, with a weighted kappa of 0.75, but with 2.6{\%} of patients having total discordance. Overall, a large proportion of clots were distal or minimally occlusive clots.",
keywords = "Computed tomography, Interobserver agreement, Pulmonary embolism, Reliability",
author = "Courtney, {D. M.} and C. Miller and H. Smithline and N. Klekowski and M. Hogg and Jeffrey Kline",
year = "2010",
month = "3",
doi = "10.1111/j.1538-7836.2009.03724.x",
language = "English (US)",
volume = "8",
pages = "533--539",
journal = "Journal of Thrombosis and Haemostasis",
issn = "1538-7933",
publisher = "Wiley-Blackwell",
number = "3",

}

TY - JOUR

T1 - Prospective multicenter assessment of interobserver agreement for radiologist interpretation of multidetector computerized tomographic angiography for pulmonary embolism

AU - Courtney, D. M.

AU - Miller, C.

AU - Smithline, H.

AU - Klekowski, N.

AU - Hogg, M.

AU - Kline, Jeffrey

PY - 2010/3

Y1 - 2010/3

N2 - Background: Emergency physicians rely on the interpretation of radiologists to diagnose and exclude pulmonary embolism (PE) on the basis of computerized tomographic pulmonary angiography (CTPA). Few data exist regarding the interobserver reliability of this endpoint. Objective: To quantify the degree of agreement in CTPA interpretation between four academic hospitals and an independent reference reading (IRR) laboratory. Methods: Hospitalized and emergency department patients who had one predefined symptom and sign of PE and underwent 64-slice CTPA were enrolled from four academic hospitals. CTPA results as interpreted by board-certified radiologists from the hospitals were compared against those from the IRR laboratory. CTPAs were read as indeterminate, PE- or PE+, and percentage obstruction was computed by the IRR laboratory, using a published method. Agreement was calculated with weighted Cohen's kappa. Results: We enrolled 492 subjects (63% female, age 54 ± 1 years, and 16.7% PE+ at the site hospitals). Overall agreement was 429/492 (87.2%; 95% confidence interval 83.9-90.0). We observed 13 cases (2.6%) of complete discordance, where one reading was PE+ and the other reading was PE-. Weighted agreement was 92.3%, with kappa = 0.75. The median percentage obstruction for all patients was 9% (25th-75th percentile interquartile range: 5% to - 30%). For CTPAs interpreted at the site hospitals as PE- or indeterminate but read as PE+ by the IRR laboratory, the median of percentage obstruction was 6% (4-7%). Conclusion: We found in this sample a good level of agreement, with a weighted kappa of 0.75, but with 2.6% of patients having total discordance. Overall, a large proportion of clots were distal or minimally occlusive clots.

AB - Background: Emergency physicians rely on the interpretation of radiologists to diagnose and exclude pulmonary embolism (PE) on the basis of computerized tomographic pulmonary angiography (CTPA). Few data exist regarding the interobserver reliability of this endpoint. Objective: To quantify the degree of agreement in CTPA interpretation between four academic hospitals and an independent reference reading (IRR) laboratory. Methods: Hospitalized and emergency department patients who had one predefined symptom and sign of PE and underwent 64-slice CTPA were enrolled from four academic hospitals. CTPA results as interpreted by board-certified radiologists from the hospitals were compared against those from the IRR laboratory. CTPAs were read as indeterminate, PE- or PE+, and percentage obstruction was computed by the IRR laboratory, using a published method. Agreement was calculated with weighted Cohen's kappa. Results: We enrolled 492 subjects (63% female, age 54 ± 1 years, and 16.7% PE+ at the site hospitals). Overall agreement was 429/492 (87.2%; 95% confidence interval 83.9-90.0). We observed 13 cases (2.6%) of complete discordance, where one reading was PE+ and the other reading was PE-. Weighted agreement was 92.3%, with kappa = 0.75. The median percentage obstruction for all patients was 9% (25th-75th percentile interquartile range: 5% to - 30%). For CTPAs interpreted at the site hospitals as PE- or indeterminate but read as PE+ by the IRR laboratory, the median of percentage obstruction was 6% (4-7%). Conclusion: We found in this sample a good level of agreement, with a weighted kappa of 0.75, but with 2.6% of patients having total discordance. Overall, a large proportion of clots were distal or minimally occlusive clots.

KW - Computed tomography

KW - Interobserver agreement

KW - Pulmonary embolism

KW - Reliability

UR - http://www.scopus.com/inward/record.url?scp=77949284445&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77949284445&partnerID=8YFLogxK

U2 - 10.1111/j.1538-7836.2009.03724.x

DO - 10.1111/j.1538-7836.2009.03724.x

M3 - Article

VL - 8

SP - 533

EP - 539

JO - Journal of Thrombosis and Haemostasis

JF - Journal of Thrombosis and Haemostasis

SN - 1538-7933

IS - 3

ER -