Reliability and validity of MRI-based automated volumetry software relative to auto-assisted manual measurement of subcortical structures in HIV-infected patients from a multisite study

Jeffrey Dewey, George Hana, Troy Russell, Jared Price, Daniel McCaffrey, Jaroslaw Harezlak, Ekta Sem, Joy C. Anyanwu, Charles R. Guttmann, Bradford Navia, Ronald Cohen, David F. Tate

Research output: Contribution to journalArticle

84 Citations (Scopus)

Abstract

The automated volumetric output of FreeSurfer and Individual Brain Atlases using Statistical Parametric Mapping (IBASPM), two widely used and well published software packages, was examined for accuracy and consistency relative to auto-assisted manual (AAM) tracings (i.e., manual correction of automated output) when measuring the caudate, putamen, amygdala, and hippocampus in the baseline scans of 120 HIV-infected patients (86.7% male, 47.3 ± 6.3. y.o., mean HIV duration 12.0 ± 6.3. years) from the NIH-funded HIV Neuroimaging Consortium (HIVNC) cohort. The data was examined for accuracy and consistency relative to auto-assisted manual tracing, and construct validity was assessed by correlating automated and AAM volumetric measures with relevant clinical measures of HIV progression. When results were averaged across all patients in the eight structures examined, FreeSurfer achieved lower absolute volume difference in five, higher sensitivity in seven, and higher spatial overlap in all eight structures. Additionally, FreeSurfer results exhibited less variability in all measures. Output from both methods identified discrepant correlations with clinical measures of HIV progression relative to AAM segmented data. Overall, FreeSurfer proved more effective in the context of subcortical volumetry in HIV-patients, particularly in a multisite cohort study such as this. These findings emphasize that regardless of the automated method used, visual inspection of segmentation output, along with manual correction if necessary, remains critical to ensuring the validity of reported results.

Original languageEnglish
Pages (from-to)1334-1344
Number of pages11
JournalNeuroImage
Volume51
Issue number4
DOIs
StatePublished - Jul 2010

Fingerprint

Reproducibility of Results
Software
HIV
Atlases
Putamen
Amygdala
Neuroimaging
Hippocampus
Cohort Studies
Brain

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology

Cite this

Reliability and validity of MRI-based automated volumetry software relative to auto-assisted manual measurement of subcortical structures in HIV-infected patients from a multisite study. / Dewey, Jeffrey; Hana, George; Russell, Troy; Price, Jared; McCaffrey, Daniel; Harezlak, Jaroslaw; Sem, Ekta; Anyanwu, Joy C.; Guttmann, Charles R.; Navia, Bradford; Cohen, Ronald; Tate, David F.

In: NeuroImage, Vol. 51, No. 4, 07.2010, p. 1334-1344.

Research output: Contribution to journalArticle

Dewey, Jeffrey ; Hana, George ; Russell, Troy ; Price, Jared ; McCaffrey, Daniel ; Harezlak, Jaroslaw ; Sem, Ekta ; Anyanwu, Joy C. ; Guttmann, Charles R. ; Navia, Bradford ; Cohen, Ronald ; Tate, David F. / Reliability and validity of MRI-based automated volumetry software relative to auto-assisted manual measurement of subcortical structures in HIV-infected patients from a multisite study. In: NeuroImage. 2010 ; Vol. 51, No. 4. pp. 1334-1344.
@article{18b8ca91217742c39ba8e2fc4fb7ace3,
title = "Reliability and validity of MRI-based automated volumetry software relative to auto-assisted manual measurement of subcortical structures in HIV-infected patients from a multisite study",
abstract = "The automated volumetric output of FreeSurfer and Individual Brain Atlases using Statistical Parametric Mapping (IBASPM), two widely used and well published software packages, was examined for accuracy and consistency relative to auto-assisted manual (AAM) tracings (i.e., manual correction of automated output) when measuring the caudate, putamen, amygdala, and hippocampus in the baseline scans of 120 HIV-infected patients (86.7{\%} male, 47.3 ± 6.3. y.o., mean HIV duration 12.0 ± 6.3. years) from the NIH-funded HIV Neuroimaging Consortium (HIVNC) cohort. The data was examined for accuracy and consistency relative to auto-assisted manual tracing, and construct validity was assessed by correlating automated and AAM volumetric measures with relevant clinical measures of HIV progression. When results were averaged across all patients in the eight structures examined, FreeSurfer achieved lower absolute volume difference in five, higher sensitivity in seven, and higher spatial overlap in all eight structures. Additionally, FreeSurfer results exhibited less variability in all measures. Output from both methods identified discrepant correlations with clinical measures of HIV progression relative to AAM segmented data. Overall, FreeSurfer proved more effective in the context of subcortical volumetry in HIV-patients, particularly in a multisite cohort study such as this. These findings emphasize that regardless of the automated method used, visual inspection of segmentation output, along with manual correction if necessary, remains critical to ensuring the validity of reported results.",
author = "Jeffrey Dewey and George Hana and Troy Russell and Jared Price and Daniel McCaffrey and Jaroslaw Harezlak and Ekta Sem and Anyanwu, {Joy C.} and Guttmann, {Charles R.} and Bradford Navia and Ronald Cohen and Tate, {David F.}",
year = "2010",
month = "7",
doi = "10.1016/j.neuroimage.2010.03.033",
language = "English",
volume = "51",
pages = "1334--1344",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",
number = "4",

}

TY - JOUR

T1 - Reliability and validity of MRI-based automated volumetry software relative to auto-assisted manual measurement of subcortical structures in HIV-infected patients from a multisite study

AU - Dewey, Jeffrey

AU - Hana, George

AU - Russell, Troy

AU - Price, Jared

AU - McCaffrey, Daniel

AU - Harezlak, Jaroslaw

AU - Sem, Ekta

AU - Anyanwu, Joy C.

AU - Guttmann, Charles R.

AU - Navia, Bradford

AU - Cohen, Ronald

AU - Tate, David F.

PY - 2010/7

Y1 - 2010/7

N2 - The automated volumetric output of FreeSurfer and Individual Brain Atlases using Statistical Parametric Mapping (IBASPM), two widely used and well published software packages, was examined for accuracy and consistency relative to auto-assisted manual (AAM) tracings (i.e., manual correction of automated output) when measuring the caudate, putamen, amygdala, and hippocampus in the baseline scans of 120 HIV-infected patients (86.7% male, 47.3 ± 6.3. y.o., mean HIV duration 12.0 ± 6.3. years) from the NIH-funded HIV Neuroimaging Consortium (HIVNC) cohort. The data was examined for accuracy and consistency relative to auto-assisted manual tracing, and construct validity was assessed by correlating automated and AAM volumetric measures with relevant clinical measures of HIV progression. When results were averaged across all patients in the eight structures examined, FreeSurfer achieved lower absolute volume difference in five, higher sensitivity in seven, and higher spatial overlap in all eight structures. Additionally, FreeSurfer results exhibited less variability in all measures. Output from both methods identified discrepant correlations with clinical measures of HIV progression relative to AAM segmented data. Overall, FreeSurfer proved more effective in the context of subcortical volumetry in HIV-patients, particularly in a multisite cohort study such as this. These findings emphasize that regardless of the automated method used, visual inspection of segmentation output, along with manual correction if necessary, remains critical to ensuring the validity of reported results.

AB - The automated volumetric output of FreeSurfer and Individual Brain Atlases using Statistical Parametric Mapping (IBASPM), two widely used and well published software packages, was examined for accuracy and consistency relative to auto-assisted manual (AAM) tracings (i.e., manual correction of automated output) when measuring the caudate, putamen, amygdala, and hippocampus in the baseline scans of 120 HIV-infected patients (86.7% male, 47.3 ± 6.3. y.o., mean HIV duration 12.0 ± 6.3. years) from the NIH-funded HIV Neuroimaging Consortium (HIVNC) cohort. The data was examined for accuracy and consistency relative to auto-assisted manual tracing, and construct validity was assessed by correlating automated and AAM volumetric measures with relevant clinical measures of HIV progression. When results were averaged across all patients in the eight structures examined, FreeSurfer achieved lower absolute volume difference in five, higher sensitivity in seven, and higher spatial overlap in all eight structures. Additionally, FreeSurfer results exhibited less variability in all measures. Output from both methods identified discrepant correlations with clinical measures of HIV progression relative to AAM segmented data. Overall, FreeSurfer proved more effective in the context of subcortical volumetry in HIV-patients, particularly in a multisite cohort study such as this. These findings emphasize that regardless of the automated method used, visual inspection of segmentation output, along with manual correction if necessary, remains critical to ensuring the validity of reported results.

UR - http://www.scopus.com/inward/record.url?scp=77952885506&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77952885506&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2010.03.033

DO - 10.1016/j.neuroimage.2010.03.033

M3 - Article

C2 - 20338250

AN - SCOPUS:77952885506

VL - 51

SP - 1334

EP - 1344

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

IS - 4

ER -