Statistical visualization for assessing performance of methods for safety surveillance using electronic databases

Xiaochun Li, Siu Hui, Patrick Ryan, Marc Rosenman, Marc Overhage

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Purpose: The success of an epidemiological study for drug safety surveillance or comparative effectiveness depends largely on design and analysis strategies besides data quality. The Observational Medical Outcomes Partnership (OMOP) methods community implemented a collection of statistical methods with extensive parameters allowing a wide variety of designs and analyses. Our objective was to develop a visualization tool to explore which parameter settings may enable better predictive properties for a given method in a database. Methods: Performance measures were produced for each setting, including sensitivity (recall), specificity (1-FPR), AUC, MAP, and Pk. Multiple regressions with relevant parameters as main effects were run for performance measures on all test cases and subgroups. Heatmaps with sequential palettes to indicate the parameters' impacts on performance measures were generated based on matrices of the standardized coefficients (t-statistics) by parameter settings and test case subgroups. Results: Heatmaps help researchers to explore design and analysis options of methods for evaluating a variety of drug-outcome relationships and also to explore data issues. Conclusions: Statistical visualization through heatmaps is a useful tool for summarizing and presenting method performance results and for the exploration of the parameter settings for method performance characteristics and data limitations.

Original languageEnglish
Pages (from-to)503-509
Number of pages7
JournalPharmacoepidemiology and Drug Safety
Volume22
Issue number5
DOIs
StatePublished - May 2013

Fingerprint

Databases
Safety
Pharmaceutical Preparations
Area Under Curve
Epidemiologic Studies
Research Personnel
Sensitivity and Specificity

Keywords

  • AUC
  • Electronic observational databases
  • Pharmacoepidemiology
  • Safety surveillance
  • Sensitivity (recall)
  • Specificity (FPR)
  • Statistical visualization

ASJC Scopus subject areas

  • Pharmacology (medical)
  • Epidemiology

Cite this

Statistical visualization for assessing performance of methods for safety surveillance using electronic databases. / Li, Xiaochun; Hui, Siu; Ryan, Patrick; Rosenman, Marc; Overhage, Marc.

In: Pharmacoepidemiology and Drug Safety, Vol. 22, No. 5, 05.2013, p. 503-509.

Research output: Contribution to journalArticle

@article{b3550714bbda4249b547fda66d98b17f,
title = "Statistical visualization for assessing performance of methods for safety surveillance using electronic databases",
abstract = "Purpose: The success of an epidemiological study for drug safety surveillance or comparative effectiveness depends largely on design and analysis strategies besides data quality. The Observational Medical Outcomes Partnership (OMOP) methods community implemented a collection of statistical methods with extensive parameters allowing a wide variety of designs and analyses. Our objective was to develop a visualization tool to explore which parameter settings may enable better predictive properties for a given method in a database. Methods: Performance measures were produced for each setting, including sensitivity (recall), specificity (1-FPR), AUC, MAP, and Pk. Multiple regressions with relevant parameters as main effects were run for performance measures on all test cases and subgroups. Heatmaps with sequential palettes to indicate the parameters' impacts on performance measures were generated based on matrices of the standardized coefficients (t-statistics) by parameter settings and test case subgroups. Results: Heatmaps help researchers to explore design and analysis options of methods for evaluating a variety of drug-outcome relationships and also to explore data issues. Conclusions: Statistical visualization through heatmaps is a useful tool for summarizing and presenting method performance results and for the exploration of the parameter settings for method performance characteristics and data limitations.",
keywords = "AUC, Electronic observational databases, Pharmacoepidemiology, Safety surveillance, Sensitivity (recall), Specificity (FPR), Statistical visualization",
author = "Xiaochun Li and Siu Hui and Patrick Ryan and Marc Rosenman and Marc Overhage",
year = "2013",
month = "5",
doi = "10.1002/pds.3419",
language = "English",
volume = "22",
pages = "503--509",
journal = "Pharmacoepidemiology and Drug Safety",
issn = "1053-8569",
publisher = "John Wiley and Sons Ltd",
number = "5",

}

TY - JOUR

T1 - Statistical visualization for assessing performance of methods for safety surveillance using electronic databases

AU - Li, Xiaochun

AU - Hui, Siu

AU - Ryan, Patrick

AU - Rosenman, Marc

AU - Overhage, Marc

PY - 2013/5

Y1 - 2013/5

N2 - Purpose: The success of an epidemiological study for drug safety surveillance or comparative effectiveness depends largely on design and analysis strategies besides data quality. The Observational Medical Outcomes Partnership (OMOP) methods community implemented a collection of statistical methods with extensive parameters allowing a wide variety of designs and analyses. Our objective was to develop a visualization tool to explore which parameter settings may enable better predictive properties for a given method in a database. Methods: Performance measures were produced for each setting, including sensitivity (recall), specificity (1-FPR), AUC, MAP, and Pk. Multiple regressions with relevant parameters as main effects were run for performance measures on all test cases and subgroups. Heatmaps with sequential palettes to indicate the parameters' impacts on performance measures were generated based on matrices of the standardized coefficients (t-statistics) by parameter settings and test case subgroups. Results: Heatmaps help researchers to explore design and analysis options of methods for evaluating a variety of drug-outcome relationships and also to explore data issues. Conclusions: Statistical visualization through heatmaps is a useful tool for summarizing and presenting method performance results and for the exploration of the parameter settings for method performance characteristics and data limitations.

AB - Purpose: The success of an epidemiological study for drug safety surveillance or comparative effectiveness depends largely on design and analysis strategies besides data quality. The Observational Medical Outcomes Partnership (OMOP) methods community implemented a collection of statistical methods with extensive parameters allowing a wide variety of designs and analyses. Our objective was to develop a visualization tool to explore which parameter settings may enable better predictive properties for a given method in a database. Methods: Performance measures were produced for each setting, including sensitivity (recall), specificity (1-FPR), AUC, MAP, and Pk. Multiple regressions with relevant parameters as main effects were run for performance measures on all test cases and subgroups. Heatmaps with sequential palettes to indicate the parameters' impacts on performance measures were generated based on matrices of the standardized coefficients (t-statistics) by parameter settings and test case subgroups. Results: Heatmaps help researchers to explore design and analysis options of methods for evaluating a variety of drug-outcome relationships and also to explore data issues. Conclusions: Statistical visualization through heatmaps is a useful tool for summarizing and presenting method performance results and for the exploration of the parameter settings for method performance characteristics and data limitations.

KW - AUC

KW - Electronic observational databases

KW - Pharmacoepidemiology

KW - Safety surveillance

KW - Sensitivity (recall)

KW - Specificity (FPR)

KW - Statistical visualization

UR - http://www.scopus.com/inward/record.url?scp=84877632020&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84877632020&partnerID=8YFLogxK

U2 - 10.1002/pds.3419

DO - 10.1002/pds.3419

M3 - Article

C2 - 23408560

AN - SCOPUS:84877632020

VL - 22

SP - 503

EP - 509

JO - Pharmacoepidemiology and Drug Safety

JF - Pharmacoepidemiology and Drug Safety

SN - 1053-8569

IS - 5

ER -