Impact of training method on the robustness of the visual assessment of 18F-florbetaben PET scans: Results from a phase-3 study

John Seibyl, Ana M. Catafau, Henryk Barthel, Kenji Ishii, Christopher C. Rowe, James B. Leverenz, Bernardino Ghetti, James W. Ironside, Masaki Takao, Hiroyasu Akatsu, Shigeo Murayama, Santiago Bullich, Andre Mueller, Norman Koglin, Walter J. Schulz-Schaeffer, Anja Hoffmann, Marwan N. Sabbagh, Andrew W. Stephens, Osama Sabri

Research output: Contribution to journalArticle

30 Citations (Scopus)

Abstract

Training for accurate image interpretation is essential for the clinical use of β-amyloid PET imaging, but the role of interpreter training and the accuracy of the algorithm for routine visual assessment of florbetaben PET scans are unclear. The aim of this study was to test the robustness of the visual assessment method for florbetaben scans, comparing efficacy readouts across different interpreters and training methods and against a histopathology standard of truth (SoT). Methods: Analysis was based on data from an international open-label, nonrandomized, multicenter phase-3 study in patients with or without dementia (ClinicalTrials.gov: NCT01020838). Florbetaben scans were assessed visually and quantitatively, and results were compared with amyloid plaque scores. For visual assessment, either in-person training (n = 3 expert interpreters) or an electronic training method (n = 5 naïve interpreters) was used. Brain samples from participants who died during the study were used to determine the histopathologic SoT using Bielschowsky silver staining (BSS) and immunohistochemistry for β-amyloid plaques. Results: Data were available from 82 patients who died and underwent postmortem histopathology. When visual assessment results were compared with BSS 1 immunohistochemistry as SoT, median sensitivity was 98.2% for the in-person-trained interpreters and 96.4% for the e-trained interpreters, and median specificity was 92.3% and 88.5%, respectively. Median accuracy was 95.1% and 91.5%, respectively. On the basis of BSS only as the SoT, median sensitivity was 98.1% and 96.2%, respectively; median specificity was 80.0% and 76.7%, respectively; and median accuracy was 91.5% and 86.6%, respectively. Interinterpreter agreement (Fleiss κ) was excellent (0.89) for in-person-trained interpreters and very good (0.71) for e-trained interpreters. Median intrainterpreter agreement was 0.9 for both in person-trained and e-trained interpreters. Visual and quantitative assessments were concordant in 88.9% of scans for in-person-trained interpreters and in 87.7% of scans for e-trained interpreters. Conclusion: Visual assessment of florbetaben images was robust in challenging scans from elderly end-of-life individuals. Sensitivity, specificity, and interinterpreter agreement were high, independent of expertise and training method. Visual assessment was accurate and reliable for detection of plaques using BSS and immunohistochemistry and well correlated with quantitative assessments.

Original languageEnglish (US)
Pages (from-to)900-906
Number of pages7
JournalJournal of Nuclear Medicine
Volume57
Issue number6
DOIs
StatePublished - Jun 1 2016

Fingerprint

Silver Staining
Positron-Emission Tomography
Immunohistochemistry
Amyloid Plaques
Amyloid
Dementia
4-(N-methylamino)-4'-(2-(2-(2-fluoroethoxy)ethoxy)ethoxy)stilbene
Sensitivity and Specificity
Brain

Keywords

  • Alzheimer's disease
  • Florbetaben
  • Positron-emission tomography

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Impact of training method on the robustness of the visual assessment of 18F-florbetaben PET scans : Results from a phase-3 study. / Seibyl, John; Catafau, Ana M.; Barthel, Henryk; Ishii, Kenji; Rowe, Christopher C.; Leverenz, James B.; Ghetti, Bernardino; Ironside, James W.; Takao, Masaki; Akatsu, Hiroyasu; Murayama, Shigeo; Bullich, Santiago; Mueller, Andre; Koglin, Norman; Schulz-Schaeffer, Walter J.; Hoffmann, Anja; Sabbagh, Marwan N.; Stephens, Andrew W.; Sabri, Osama.

In: Journal of Nuclear Medicine, Vol. 57, No. 6, 01.06.2016, p. 900-906.

Research output: Contribution to journalArticle

Seibyl, J, Catafau, AM, Barthel, H, Ishii, K, Rowe, CC, Leverenz, JB, Ghetti, B, Ironside, JW, Takao, M, Akatsu, H, Murayama, S, Bullich, S, Mueller, A, Koglin, N, Schulz-Schaeffer, WJ, Hoffmann, A, Sabbagh, MN, Stephens, AW & Sabri, O 2016, 'Impact of training method on the robustness of the visual assessment of 18F-florbetaben PET scans: Results from a phase-3 study', Journal of Nuclear Medicine, vol. 57, no. 6, pp. 900-906. https://doi.org/10.2967/jnumed.115.161927
Seibyl, John ; Catafau, Ana M. ; Barthel, Henryk ; Ishii, Kenji ; Rowe, Christopher C. ; Leverenz, James B. ; Ghetti, Bernardino ; Ironside, James W. ; Takao, Masaki ; Akatsu, Hiroyasu ; Murayama, Shigeo ; Bullich, Santiago ; Mueller, Andre ; Koglin, Norman ; Schulz-Schaeffer, Walter J. ; Hoffmann, Anja ; Sabbagh, Marwan N. ; Stephens, Andrew W. ; Sabri, Osama. / Impact of training method on the robustness of the visual assessment of 18F-florbetaben PET scans : Results from a phase-3 study. In: Journal of Nuclear Medicine. 2016 ; Vol. 57, No. 6. pp. 900-906.
@article{084ea4b3f2bc441685cd151062a242ef,
title = "Impact of training method on the robustness of the visual assessment of 18F-florbetaben PET scans: Results from a phase-3 study",
abstract = "Training for accurate image interpretation is essential for the clinical use of β-amyloid PET imaging, but the role of interpreter training and the accuracy of the algorithm for routine visual assessment of florbetaben PET scans are unclear. The aim of this study was to test the robustness of the visual assessment method for florbetaben scans, comparing efficacy readouts across different interpreters and training methods and against a histopathology standard of truth (SoT). Methods: Analysis was based on data from an international open-label, nonrandomized, multicenter phase-3 study in patients with or without dementia (ClinicalTrials.gov: NCT01020838). Florbetaben scans were assessed visually and quantitatively, and results were compared with amyloid plaque scores. For visual assessment, either in-person training (n = 3 expert interpreters) or an electronic training method (n = 5 na{\"i}ve interpreters) was used. Brain samples from participants who died during the study were used to determine the histopathologic SoT using Bielschowsky silver staining (BSS) and immunohistochemistry for β-amyloid plaques. Results: Data were available from 82 patients who died and underwent postmortem histopathology. When visual assessment results were compared with BSS 1 immunohistochemistry as SoT, median sensitivity was 98.2{\%} for the in-person-trained interpreters and 96.4{\%} for the e-trained interpreters, and median specificity was 92.3{\%} and 88.5{\%}, respectively. Median accuracy was 95.1{\%} and 91.5{\%}, respectively. On the basis of BSS only as the SoT, median sensitivity was 98.1{\%} and 96.2{\%}, respectively; median specificity was 80.0{\%} and 76.7{\%}, respectively; and median accuracy was 91.5{\%} and 86.6{\%}, respectively. Interinterpreter agreement (Fleiss κ) was excellent (0.89) for in-person-trained interpreters and very good (0.71) for e-trained interpreters. Median intrainterpreter agreement was 0.9 for both in person-trained and e-trained interpreters. Visual and quantitative assessments were concordant in 88.9{\%} of scans for in-person-trained interpreters and in 87.7{\%} of scans for e-trained interpreters. Conclusion: Visual assessment of florbetaben images was robust in challenging scans from elderly end-of-life individuals. Sensitivity, specificity, and interinterpreter agreement were high, independent of expertise and training method. Visual assessment was accurate and reliable for detection of plaques using BSS and immunohistochemistry and well correlated with quantitative assessments.",
keywords = "Alzheimer's disease, Florbetaben, Positron-emission tomography",
author = "John Seibyl and Catafau, {Ana M.} and Henryk Barthel and Kenji Ishii and Rowe, {Christopher C.} and Leverenz, {James B.} and Bernardino Ghetti and Ironside, {James W.} and Masaki Takao and Hiroyasu Akatsu and Shigeo Murayama and Santiago Bullich and Andre Mueller and Norman Koglin and Schulz-Schaeffer, {Walter J.} and Anja Hoffmann and Sabbagh, {Marwan N.} and Stephens, {Andrew W.} and Osama Sabri",
year = "2016",
month = "6",
day = "1",
doi = "10.2967/jnumed.115.161927",
language = "English (US)",
volume = "57",
pages = "900--906",
journal = "Journal of Nuclear Medicine",
issn = "0161-5505",
publisher = "Society of Nuclear Medicine Inc.",
number = "6",

}

TY - JOUR

T1 - Impact of training method on the robustness of the visual assessment of 18F-florbetaben PET scans

T2 - Results from a phase-3 study

AU - Seibyl, John

AU - Catafau, Ana M.

AU - Barthel, Henryk

AU - Ishii, Kenji

AU - Rowe, Christopher C.

AU - Leverenz, James B.

AU - Ghetti, Bernardino

AU - Ironside, James W.

AU - Takao, Masaki

AU - Akatsu, Hiroyasu

AU - Murayama, Shigeo

AU - Bullich, Santiago

AU - Mueller, Andre

AU - Koglin, Norman

AU - Schulz-Schaeffer, Walter J.

AU - Hoffmann, Anja

AU - Sabbagh, Marwan N.

AU - Stephens, Andrew W.

AU - Sabri, Osama

PY - 2016/6/1

Y1 - 2016/6/1

N2 - Training for accurate image interpretation is essential for the clinical use of β-amyloid PET imaging, but the role of interpreter training and the accuracy of the algorithm for routine visual assessment of florbetaben PET scans are unclear. The aim of this study was to test the robustness of the visual assessment method for florbetaben scans, comparing efficacy readouts across different interpreters and training methods and against a histopathology standard of truth (SoT). Methods: Analysis was based on data from an international open-label, nonrandomized, multicenter phase-3 study in patients with or without dementia (ClinicalTrials.gov: NCT01020838). Florbetaben scans were assessed visually and quantitatively, and results were compared with amyloid plaque scores. For visual assessment, either in-person training (n = 3 expert interpreters) or an electronic training method (n = 5 naïve interpreters) was used. Brain samples from participants who died during the study were used to determine the histopathologic SoT using Bielschowsky silver staining (BSS) and immunohistochemistry for β-amyloid plaques. Results: Data were available from 82 patients who died and underwent postmortem histopathology. When visual assessment results were compared with BSS 1 immunohistochemistry as SoT, median sensitivity was 98.2% for the in-person-trained interpreters and 96.4% for the e-trained interpreters, and median specificity was 92.3% and 88.5%, respectively. Median accuracy was 95.1% and 91.5%, respectively. On the basis of BSS only as the SoT, median sensitivity was 98.1% and 96.2%, respectively; median specificity was 80.0% and 76.7%, respectively; and median accuracy was 91.5% and 86.6%, respectively. Interinterpreter agreement (Fleiss κ) was excellent (0.89) for in-person-trained interpreters and very good (0.71) for e-trained interpreters. Median intrainterpreter agreement was 0.9 for both in person-trained and e-trained interpreters. Visual and quantitative assessments were concordant in 88.9% of scans for in-person-trained interpreters and in 87.7% of scans for e-trained interpreters. Conclusion: Visual assessment of florbetaben images was robust in challenging scans from elderly end-of-life individuals. Sensitivity, specificity, and interinterpreter agreement were high, independent of expertise and training method. Visual assessment was accurate and reliable for detection of plaques using BSS and immunohistochemistry and well correlated with quantitative assessments.

AB - Training for accurate image interpretation is essential for the clinical use of β-amyloid PET imaging, but the role of interpreter training and the accuracy of the algorithm for routine visual assessment of florbetaben PET scans are unclear. The aim of this study was to test the robustness of the visual assessment method for florbetaben scans, comparing efficacy readouts across different interpreters and training methods and against a histopathology standard of truth (SoT). Methods: Analysis was based on data from an international open-label, nonrandomized, multicenter phase-3 study in patients with or without dementia (ClinicalTrials.gov: NCT01020838). Florbetaben scans were assessed visually and quantitatively, and results were compared with amyloid plaque scores. For visual assessment, either in-person training (n = 3 expert interpreters) or an electronic training method (n = 5 naïve interpreters) was used. Brain samples from participants who died during the study were used to determine the histopathologic SoT using Bielschowsky silver staining (BSS) and immunohistochemistry for β-amyloid plaques. Results: Data were available from 82 patients who died and underwent postmortem histopathology. When visual assessment results were compared with BSS 1 immunohistochemistry as SoT, median sensitivity was 98.2% for the in-person-trained interpreters and 96.4% for the e-trained interpreters, and median specificity was 92.3% and 88.5%, respectively. Median accuracy was 95.1% and 91.5%, respectively. On the basis of BSS only as the SoT, median sensitivity was 98.1% and 96.2%, respectively; median specificity was 80.0% and 76.7%, respectively; and median accuracy was 91.5% and 86.6%, respectively. Interinterpreter agreement (Fleiss κ) was excellent (0.89) for in-person-trained interpreters and very good (0.71) for e-trained interpreters. Median intrainterpreter agreement was 0.9 for both in person-trained and e-trained interpreters. Visual and quantitative assessments were concordant in 88.9% of scans for in-person-trained interpreters and in 87.7% of scans for e-trained interpreters. Conclusion: Visual assessment of florbetaben images was robust in challenging scans from elderly end-of-life individuals. Sensitivity, specificity, and interinterpreter agreement were high, independent of expertise and training method. Visual assessment was accurate and reliable for detection of plaques using BSS and immunohistochemistry and well correlated with quantitative assessments.

KW - Alzheimer's disease

KW - Florbetaben

KW - Positron-emission tomography

UR - http://www.scopus.com/inward/record.url?scp=84973664483&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973664483&partnerID=8YFLogxK

U2 - 10.2967/jnumed.115.161927

DO - 10.2967/jnumed.115.161927

M3 - Article

C2 - 26823561

AN - SCOPUS:84973664483

VL - 57

SP - 900

EP - 906

JO - Journal of Nuclear Medicine

JF - Journal of Nuclear Medicine

SN - 0161-5505

IS - 6

ER -