Evaluating the surgery literature: Can standardizing peer-review today predict manuscript impact tomorrow?

Julie A. Sosa, Pritesh Mehta, Daniel C. Thomas, Gretchen Berland, Cary Gross, Robert L. McNamara, Ronnie Rosenthal, Robert Udelsman, Dawn M. Bravata, Sanziana A. Roman

Research output: Contribution to journalArticle

6 Scopus citations


OBJECTIVE: Evidence-based surgery is predicated on the quality of published literature. We measured the quality of surgery manuscripts selected by peer review and identified predictors of excellence. METHODS: One hundred twenty clinical surgery manuscripts were randomly selected from 1998 in 5 eminent peer-reviewed surgery and medical journals. Manuscripts were blinded for author, institution, and journal of origin. Four surgeons and 4 methodologists evaluated the quality using novel instruments based on subject selection, study protocol, statistical analysis/inference, intervention description, outcome assessments, and results presentation. Predictors of quality and impact factor were identified using bivariate and multivariate regression. RESULTS: Oncology was the most common subject (26%), followed by general surgery/gastrointestinal (24%). The average number of study subjects was 417; the majority of manuscripts were American (53%), from a single institution (59%). Eighteen percent had a statistician author. Mean number of citations was 128. Surgery manuscripts from medical, compared with surgery journals, had better total quality scores (3.8 vs. 5.2, P < 0.001). They had more subjects and were more likely to have a statistician as coauthor (43% vs. 10%, P < 0.001), multi-institutional, international collaboration (30% vs. 8%, P < 0.001), and higher citation index (mean: 350 vs. 54, P < 0.001). They were more often foreign (70% vs. 40%, P < 0.001). Independent predictors of quality were having a statistician coauthor, study funding, European origin, and more study subjects. Quality assessment using our instruments predicted the number of citations after 10 years (P < 0.01), along with having a statistician coauthor, international multi-institutional collaboration, and more subjects. CONCLUSION: The quality of surgery manuscripts can be improved by including a statistician as coauthor, with efforts directed toward implementing multi-institutional/interdisciplinary trials. Peer-review across journals can be standardized through the use of instruments measuring methodologic and clinical quality.

Original languageEnglish (US)
Pages (from-to)152-158
Number of pages7
JournalAnnals of surgery
Issue number1
StatePublished - Jul 1 2009

ASJC Scopus subject areas

  • Surgery

Fingerprint Dive into the research topics of 'Evaluating the surgery literature: Can standardizing peer-review today predict manuscript impact tomorrow?'. Together they form a unique fingerprint.

  • Cite this

    Sosa, J. A., Mehta, P., Thomas, D. C., Berland, G., Gross, C., McNamara, R. L., Rosenthal, R., Udelsman, R., Bravata, D. M., & Roman, S. A. (2009). Evaluating the surgery literature: Can standardizing peer-review today predict manuscript impact tomorrow? Annals of surgery, 250(1), 152-158. https://doi.org/10.1097/SLA.0b013e3181ad8905