Evaluating the surgery literature: Can standardizing peer-review today predict manuscript impact tomorrow?

Julie A. Sosa, Pritesh Mehta, Daniel C. Thomas, Gretchen Berland, Cary Gross, Robert L. McNamara, Ronnie Rosenthal, Robert Udelsman, Dawn Bravata, Sanziana A. Roman

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

OBJECTIVE: Evidence-based surgery is predicated on the quality of published literature. We measured the quality of surgery manuscripts selected by peer review and identified predictors of excellence. METHODS: One hundred twenty clinical surgery manuscripts were randomly selected from 1998 in 5 eminent peer-reviewed surgery and medical journals. Manuscripts were blinded for author, institution, and journal of origin. Four surgeons and 4 methodologists evaluated the quality using novel instruments based on subject selection, study protocol, statistical analysis/inference, intervention description, outcome assessments, and results presentation. Predictors of quality and impact factor were identified using bivariate and multivariate regression. RESULTS: Oncology was the most common subject (26%), followed by general surgery/gastrointestinal (24%). The average number of study subjects was 417; the majority of manuscripts were American (53%), from a single institution (59%). Eighteen percent had a statistician author. Mean number of citations was 128. Surgery manuscripts from medical, compared with surgery journals, had better total quality scores (3.8 vs. 5.2, P < 0.001). They had more subjects and were more likely to have a statistician as coauthor (43% vs. 10%, P < 0.001), multi-institutional, international collaboration (30% vs. 8%, P < 0.001), and higher citation index (mean: 350 vs. 54, P < 0.001). They were more often foreign (70% vs. 40%, P < 0.001). Independent predictors of quality were having a statistician coauthor, study funding, European origin, and more study subjects. Quality assessment using our instruments predicted the number of citations after 10 years (P < 0.01), along with having a statistician coauthor, international multi-institutional collaboration, and more subjects. CONCLUSION: The quality of surgery manuscripts can be improved by including a statistician as coauthor, with efforts directed toward implementing multi-institutional/interdisciplinary trials. Peer-review across journals can be standardized through the use of instruments measuring methodologic and clinical quality.

Original languageEnglish
Pages (from-to)152-158
Number of pages7
JournalAnnals of Surgery
Volume250
Issue number1
DOIs
StatePublished - Jul 2009

Fingerprint

Peer Review
Manuscripts
Medical Manuscripts
Patient Selection
Outcome Assessment (Health Care)

ASJC Scopus subject areas

  • Surgery

Cite this

Sosa, J. A., Mehta, P., Thomas, D. C., Berland, G., Gross, C., McNamara, R. L., ... Roman, S. A. (2009). Evaluating the surgery literature: Can standardizing peer-review today predict manuscript impact tomorrow? Annals of Surgery, 250(1), 152-158. https://doi.org/10.1097/SLA.0b013e3181ad8905

Evaluating the surgery literature : Can standardizing peer-review today predict manuscript impact tomorrow? / Sosa, Julie A.; Mehta, Pritesh; Thomas, Daniel C.; Berland, Gretchen; Gross, Cary; McNamara, Robert L.; Rosenthal, Ronnie; Udelsman, Robert; Bravata, Dawn; Roman, Sanziana A.

In: Annals of Surgery, Vol. 250, No. 1, 07.2009, p. 152-158.

Research output: Contribution to journalArticle

Sosa, JA, Mehta, P, Thomas, DC, Berland, G, Gross, C, McNamara, RL, Rosenthal, R, Udelsman, R, Bravata, D & Roman, SA 2009, 'Evaluating the surgery literature: Can standardizing peer-review today predict manuscript impact tomorrow?', Annals of Surgery, vol. 250, no. 1, pp. 152-158. https://doi.org/10.1097/SLA.0b013e3181ad8905
Sosa, Julie A. ; Mehta, Pritesh ; Thomas, Daniel C. ; Berland, Gretchen ; Gross, Cary ; McNamara, Robert L. ; Rosenthal, Ronnie ; Udelsman, Robert ; Bravata, Dawn ; Roman, Sanziana A. / Evaluating the surgery literature : Can standardizing peer-review today predict manuscript impact tomorrow?. In: Annals of Surgery. 2009 ; Vol. 250, No. 1. pp. 152-158.
@article{8d903dd31a6b4722b82553d7a48dc853,
title = "Evaluating the surgery literature: Can standardizing peer-review today predict manuscript impact tomorrow?",
abstract = "OBJECTIVE: Evidence-based surgery is predicated on the quality of published literature. We measured the quality of surgery manuscripts selected by peer review and identified predictors of excellence. METHODS: One hundred twenty clinical surgery manuscripts were randomly selected from 1998 in 5 eminent peer-reviewed surgery and medical journals. Manuscripts were blinded for author, institution, and journal of origin. Four surgeons and 4 methodologists evaluated the quality using novel instruments based on subject selection, study protocol, statistical analysis/inference, intervention description, outcome assessments, and results presentation. Predictors of quality and impact factor were identified using bivariate and multivariate regression. RESULTS: Oncology was the most common subject (26{\%}), followed by general surgery/gastrointestinal (24{\%}). The average number of study subjects was 417; the majority of manuscripts were American (53{\%}), from a single institution (59{\%}). Eighteen percent had a statistician author. Mean number of citations was 128. Surgery manuscripts from medical, compared with surgery journals, had better total quality scores (3.8 vs. 5.2, P < 0.001). They had more subjects and were more likely to have a statistician as coauthor (43{\%} vs. 10{\%}, P < 0.001), multi-institutional, international collaboration (30{\%} vs. 8{\%}, P < 0.001), and higher citation index (mean: 350 vs. 54, P < 0.001). They were more often foreign (70{\%} vs. 40{\%}, P < 0.001). Independent predictors of quality were having a statistician coauthor, study funding, European origin, and more study subjects. Quality assessment using our instruments predicted the number of citations after 10 years (P < 0.01), along with having a statistician coauthor, international multi-institutional collaboration, and more subjects. CONCLUSION: The quality of surgery manuscripts can be improved by including a statistician as coauthor, with efforts directed toward implementing multi-institutional/interdisciplinary trials. Peer-review across journals can be standardized through the use of instruments measuring methodologic and clinical quality.",
author = "Sosa, {Julie A.} and Pritesh Mehta and Thomas, {Daniel C.} and Gretchen Berland and Cary Gross and McNamara, {Robert L.} and Ronnie Rosenthal and Robert Udelsman and Dawn Bravata and Roman, {Sanziana A.}",
year = "2009",
month = "7",
doi = "10.1097/SLA.0b013e3181ad8905",
language = "English",
volume = "250",
pages = "152--158",
journal = "Annals of Surgery",
issn = "0003-4932",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Evaluating the surgery literature

T2 - Can standardizing peer-review today predict manuscript impact tomorrow?

AU - Sosa, Julie A.

AU - Mehta, Pritesh

AU - Thomas, Daniel C.

AU - Berland, Gretchen

AU - Gross, Cary

AU - McNamara, Robert L.

AU - Rosenthal, Ronnie

AU - Udelsman, Robert

AU - Bravata, Dawn

AU - Roman, Sanziana A.

PY - 2009/7

Y1 - 2009/7

N2 - OBJECTIVE: Evidence-based surgery is predicated on the quality of published literature. We measured the quality of surgery manuscripts selected by peer review and identified predictors of excellence. METHODS: One hundred twenty clinical surgery manuscripts were randomly selected from 1998 in 5 eminent peer-reviewed surgery and medical journals. Manuscripts were blinded for author, institution, and journal of origin. Four surgeons and 4 methodologists evaluated the quality using novel instruments based on subject selection, study protocol, statistical analysis/inference, intervention description, outcome assessments, and results presentation. Predictors of quality and impact factor were identified using bivariate and multivariate regression. RESULTS: Oncology was the most common subject (26%), followed by general surgery/gastrointestinal (24%). The average number of study subjects was 417; the majority of manuscripts were American (53%), from a single institution (59%). Eighteen percent had a statistician author. Mean number of citations was 128. Surgery manuscripts from medical, compared with surgery journals, had better total quality scores (3.8 vs. 5.2, P < 0.001). They had more subjects and were more likely to have a statistician as coauthor (43% vs. 10%, P < 0.001), multi-institutional, international collaboration (30% vs. 8%, P < 0.001), and higher citation index (mean: 350 vs. 54, P < 0.001). They were more often foreign (70% vs. 40%, P < 0.001). Independent predictors of quality were having a statistician coauthor, study funding, European origin, and more study subjects. Quality assessment using our instruments predicted the number of citations after 10 years (P < 0.01), along with having a statistician coauthor, international multi-institutional collaboration, and more subjects. CONCLUSION: The quality of surgery manuscripts can be improved by including a statistician as coauthor, with efforts directed toward implementing multi-institutional/interdisciplinary trials. Peer-review across journals can be standardized through the use of instruments measuring methodologic and clinical quality.

AB - OBJECTIVE: Evidence-based surgery is predicated on the quality of published literature. We measured the quality of surgery manuscripts selected by peer review and identified predictors of excellence. METHODS: One hundred twenty clinical surgery manuscripts were randomly selected from 1998 in 5 eminent peer-reviewed surgery and medical journals. Manuscripts were blinded for author, institution, and journal of origin. Four surgeons and 4 methodologists evaluated the quality using novel instruments based on subject selection, study protocol, statistical analysis/inference, intervention description, outcome assessments, and results presentation. Predictors of quality and impact factor were identified using bivariate and multivariate regression. RESULTS: Oncology was the most common subject (26%), followed by general surgery/gastrointestinal (24%). The average number of study subjects was 417; the majority of manuscripts were American (53%), from a single institution (59%). Eighteen percent had a statistician author. Mean number of citations was 128. Surgery manuscripts from medical, compared with surgery journals, had better total quality scores (3.8 vs. 5.2, P < 0.001). They had more subjects and were more likely to have a statistician as coauthor (43% vs. 10%, P < 0.001), multi-institutional, international collaboration (30% vs. 8%, P < 0.001), and higher citation index (mean: 350 vs. 54, P < 0.001). They were more often foreign (70% vs. 40%, P < 0.001). Independent predictors of quality were having a statistician coauthor, study funding, European origin, and more study subjects. Quality assessment using our instruments predicted the number of citations after 10 years (P < 0.01), along with having a statistician coauthor, international multi-institutional collaboration, and more subjects. CONCLUSION: The quality of surgery manuscripts can be improved by including a statistician as coauthor, with efforts directed toward implementing multi-institutional/interdisciplinary trials. Peer-review across journals can be standardized through the use of instruments measuring methodologic and clinical quality.

UR - http://www.scopus.com/inward/record.url?scp=68249125955&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=68249125955&partnerID=8YFLogxK

U2 - 10.1097/SLA.0b013e3181ad8905

DO - 10.1097/SLA.0b013e3181ad8905

M3 - Article

C2 - 19561471

AN - SCOPUS:68249125955

VL - 250

SP - 152

EP - 158

JO - Annals of Surgery

JF - Annals of Surgery

SN - 0003-4932

IS - 1

ER -