Standardized evaluation of tumor-infiltrating lymphocytes in breast cancer

results of the ring studies of the international immuno-oncology biomarker working group

Carsten Denkert, Stephan Wienert, Audrey Poterie, Sibylle Loibl, Jan Budczies, Sunil Badve, Zsuzsanna Bago-Horvath, Anita Bane, Shahinaz Bedri, Jane Brock, Ewa Chmielik, Matthias Christgen, Cecile Colpaert, Sandra Demaria, Gert van den Eynden, Giuseppe Floris, Stephen B. Fox, Dongxia Gao, Barbara Ingold Heppner, S. Rim Kim & 24 others Zuzana Kos, Hans H. Kreipe, Sunil R. Lakhani, Frederique Penault-Llorca, Giancarlo Pruneri, Nina Radosevic-Robin, David L. Rimm, Stuart J. Schnitt, Bruno V. Sinn, Peter Sinn, Nicolas Sirtaine, Sandra A. O'Toole, Giuseppe Viale, Koen van de Vijver, Roland de Wind, Gunter von Minckwitz, Frederick Klauschen, Michael Untch, Peter A. Fasching, Toralf Reimer, Karen Willard-Gallo, Stefan Michiels, Sherene Loi, Roberto Salgado

Research output: Contribution to journalArticle

58 Citations (Scopus)

Abstract

Multiple independent studies have shown that tumor-infiltrating lymphocytes (TIL) are prognostic in breast cancer with potential relevance for response to immune-checkpoint inhibitor therapy. Although many groups are currently evaluating TIL, there is no standardized system for diagnostic applications. This study reports the results of two ring studies investigating TIL conducted by the International Working Group on Immuno-oncology Biomarkers. The study aim was to determine the intraclass correlation coefficient (ICC) for evaluation of TIL by different pathologists. A total of 120 slides were evaluated by a large group of pathologists with a web-based system in ring study 1 and a more advanced software-system in ring study 2 that included an integrated feedback with standardized reference images. The predefined aim for successful ring studies 1 and 2 was an ICC above 0.7 (lower limit of 95% confidence interval (CI)). In ring study 1 the prespecified endpoint was not reached (ICC: 0.70; 95% CI: 0.62–0.78). On the basis of an analysis of sources of variation, we developed a more advanced digital image evaluation system for ring study 2, which improved the ICC to 0.89 (95% CI: 0.85–0.92). The Fleiss’ kappa value for <60 vs ≥60% TIL improved from 0.45 (ring study 1) to 0.63 in RS2 and the mean concordance improved from 88 to 92%. This large international standardization project shows that reproducible evaluation of TIL is feasible in breast cancer. This opens the way for standardized reporting of tumor immunological parameters in clinical studies and diagnostic practice. The software-guided image evaluation approach used in ring study 2 may be of value as a tool for evaluation of TIL in clinical trials and diagnostic practice. The experience gained from this approach might be applicable to the standardization of other diagnostic parameters in histopathology.Modern Pathology advance online publication, 1 July 2016; doi:10.1038/modpathol.2016.109.

Original languageEnglish (US)
JournalModern Pathology
DOIs
StateAccepted/In press - Jul 1 2016

Fingerprint

Tumor-Infiltrating Lymphocytes
Biomarkers
Breast Neoplasms
Confidence Intervals
Software
Publications
Clinical Trials
Pathology

ASJC Scopus subject areas

  • Pathology and Forensic Medicine
  • Medicine(all)

Cite this

Standardized evaluation of tumor-infiltrating lymphocytes in breast cancer : results of the ring studies of the international immuno-oncology biomarker working group. / Denkert, Carsten; Wienert, Stephan; Poterie, Audrey; Loibl, Sibylle; Budczies, Jan; Badve, Sunil; Bago-Horvath, Zsuzsanna; Bane, Anita; Bedri, Shahinaz; Brock, Jane; Chmielik, Ewa; Christgen, Matthias; Colpaert, Cecile; Demaria, Sandra; van den Eynden, Gert; Floris, Giuseppe; Fox, Stephen B.; Gao, Dongxia; Ingold Heppner, Barbara; Kim, S. Rim; Kos, Zuzana; Kreipe, Hans H.; Lakhani, Sunil R.; Penault-Llorca, Frederique; Pruneri, Giancarlo; Radosevic-Robin, Nina; Rimm, David L.; Schnitt, Stuart J.; Sinn, Bruno V.; Sinn, Peter; Sirtaine, Nicolas; O'Toole, Sandra A.; Viale, Giuseppe; van de Vijver, Koen; de Wind, Roland; von Minckwitz, Gunter; Klauschen, Frederick; Untch, Michael; Fasching, Peter A.; Reimer, Toralf; Willard-Gallo, Karen; Michiels, Stefan; Loi, Sherene; Salgado, Roberto.

In: Modern Pathology, 01.07.2016.

Research output: Contribution to journalArticle

Denkert, C, Wienert, S, Poterie, A, Loibl, S, Budczies, J, Badve, S, Bago-Horvath, Z, Bane, A, Bedri, S, Brock, J, Chmielik, E, Christgen, M, Colpaert, C, Demaria, S, van den Eynden, G, Floris, G, Fox, SB, Gao, D, Ingold Heppner, B, Kim, SR, Kos, Z, Kreipe, HH, Lakhani, SR, Penault-Llorca, F, Pruneri, G, Radosevic-Robin, N, Rimm, DL, Schnitt, SJ, Sinn, BV, Sinn, P, Sirtaine, N, O'Toole, SA, Viale, G, van de Vijver, K, de Wind, R, von Minckwitz, G, Klauschen, F, Untch, M, Fasching, PA, Reimer, T, Willard-Gallo, K, Michiels, S, Loi, S & Salgado, R 2016, 'Standardized evaluation of tumor-infiltrating lymphocytes in breast cancer: results of the ring studies of the international immuno-oncology biomarker working group', Modern Pathology. https://doi.org/10.1038/modpathol.2016.109
Denkert, Carsten ; Wienert, Stephan ; Poterie, Audrey ; Loibl, Sibylle ; Budczies, Jan ; Badve, Sunil ; Bago-Horvath, Zsuzsanna ; Bane, Anita ; Bedri, Shahinaz ; Brock, Jane ; Chmielik, Ewa ; Christgen, Matthias ; Colpaert, Cecile ; Demaria, Sandra ; van den Eynden, Gert ; Floris, Giuseppe ; Fox, Stephen B. ; Gao, Dongxia ; Ingold Heppner, Barbara ; Kim, S. Rim ; Kos, Zuzana ; Kreipe, Hans H. ; Lakhani, Sunil R. ; Penault-Llorca, Frederique ; Pruneri, Giancarlo ; Radosevic-Robin, Nina ; Rimm, David L. ; Schnitt, Stuart J. ; Sinn, Bruno V. ; Sinn, Peter ; Sirtaine, Nicolas ; O'Toole, Sandra A. ; Viale, Giuseppe ; van de Vijver, Koen ; de Wind, Roland ; von Minckwitz, Gunter ; Klauschen, Frederick ; Untch, Michael ; Fasching, Peter A. ; Reimer, Toralf ; Willard-Gallo, Karen ; Michiels, Stefan ; Loi, Sherene ; Salgado, Roberto. / Standardized evaluation of tumor-infiltrating lymphocytes in breast cancer : results of the ring studies of the international immuno-oncology biomarker working group. In: Modern Pathology. 2016.
@article{d876f4bec48e4e819499bc3b2787f85c,
title = "Standardized evaluation of tumor-infiltrating lymphocytes in breast cancer: results of the ring studies of the international immuno-oncology biomarker working group",
abstract = "Multiple independent studies have shown that tumor-infiltrating lymphocytes (TIL) are prognostic in breast cancer with potential relevance for response to immune-checkpoint inhibitor therapy. Although many groups are currently evaluating TIL, there is no standardized system for diagnostic applications. This study reports the results of two ring studies investigating TIL conducted by the International Working Group on Immuno-oncology Biomarkers. The study aim was to determine the intraclass correlation coefficient (ICC) for evaluation of TIL by different pathologists. A total of 120 slides were evaluated by a large group of pathologists with a web-based system in ring study 1 and a more advanced software-system in ring study 2 that included an integrated feedback with standardized reference images. The predefined aim for successful ring studies 1 and 2 was an ICC above 0.7 (lower limit of 95{\%} confidence interval (CI)). In ring study 1 the prespecified endpoint was not reached (ICC: 0.70; 95{\%} CI: 0.62–0.78). On the basis of an analysis of sources of variation, we developed a more advanced digital image evaluation system for ring study 2, which improved the ICC to 0.89 (95{\%} CI: 0.85–0.92). The Fleiss’ kappa value for <60 vs ≥60{\%} TIL improved from 0.45 (ring study 1) to 0.63 in RS2 and the mean concordance improved from 88 to 92{\%}. This large international standardization project shows that reproducible evaluation of TIL is feasible in breast cancer. This opens the way for standardized reporting of tumor immunological parameters in clinical studies and diagnostic practice. The software-guided image evaluation approach used in ring study 2 may be of value as a tool for evaluation of TIL in clinical trials and diagnostic practice. The experience gained from this approach might be applicable to the standardization of other diagnostic parameters in histopathology.Modern Pathology advance online publication, 1 July 2016; doi:10.1038/modpathol.2016.109.",
author = "Carsten Denkert and Stephan Wienert and Audrey Poterie and Sibylle Loibl and Jan Budczies and Sunil Badve and Zsuzsanna Bago-Horvath and Anita Bane and Shahinaz Bedri and Jane Brock and Ewa Chmielik and Matthias Christgen and Cecile Colpaert and Sandra Demaria and {van den Eynden}, Gert and Giuseppe Floris and Fox, {Stephen B.} and Dongxia Gao and {Ingold Heppner}, Barbara and Kim, {S. Rim} and Zuzana Kos and Kreipe, {Hans H.} and Lakhani, {Sunil R.} and Frederique Penault-Llorca and Giancarlo Pruneri and Nina Radosevic-Robin and Rimm, {David L.} and Schnitt, {Stuart J.} and Sinn, {Bruno V.} and Peter Sinn and Nicolas Sirtaine and O'Toole, {Sandra A.} and Giuseppe Viale and {van de Vijver}, Koen and {de Wind}, Roland and {von Minckwitz}, Gunter and Frederick Klauschen and Michael Untch and Fasching, {Peter A.} and Toralf Reimer and Karen Willard-Gallo and Stefan Michiels and Sherene Loi and Roberto Salgado",
year = "2016",
month = "7",
day = "1",
doi = "10.1038/modpathol.2016.109",
language = "English (US)",
journal = "Modern Pathology",
issn = "0893-3952",
publisher = "Nature Publishing Group",

}

TY - JOUR

T1 - Standardized evaluation of tumor-infiltrating lymphocytes in breast cancer

T2 - results of the ring studies of the international immuno-oncology biomarker working group

AU - Denkert, Carsten

AU - Wienert, Stephan

AU - Poterie, Audrey

AU - Loibl, Sibylle

AU - Budczies, Jan

AU - Badve, Sunil

AU - Bago-Horvath, Zsuzsanna

AU - Bane, Anita

AU - Bedri, Shahinaz

AU - Brock, Jane

AU - Chmielik, Ewa

AU - Christgen, Matthias

AU - Colpaert, Cecile

AU - Demaria, Sandra

AU - van den Eynden, Gert

AU - Floris, Giuseppe

AU - Fox, Stephen B.

AU - Gao, Dongxia

AU - Ingold Heppner, Barbara

AU - Kim, S. Rim

AU - Kos, Zuzana

AU - Kreipe, Hans H.

AU - Lakhani, Sunil R.

AU - Penault-Llorca, Frederique

AU - Pruneri, Giancarlo

AU - Radosevic-Robin, Nina

AU - Rimm, David L.

AU - Schnitt, Stuart J.

AU - Sinn, Bruno V.

AU - Sinn, Peter

AU - Sirtaine, Nicolas

AU - O'Toole, Sandra A.

AU - Viale, Giuseppe

AU - van de Vijver, Koen

AU - de Wind, Roland

AU - von Minckwitz, Gunter

AU - Klauschen, Frederick

AU - Untch, Michael

AU - Fasching, Peter A.

AU - Reimer, Toralf

AU - Willard-Gallo, Karen

AU - Michiels, Stefan

AU - Loi, Sherene

AU - Salgado, Roberto

PY - 2016/7/1

Y1 - 2016/7/1

N2 - Multiple independent studies have shown that tumor-infiltrating lymphocytes (TIL) are prognostic in breast cancer with potential relevance for response to immune-checkpoint inhibitor therapy. Although many groups are currently evaluating TIL, there is no standardized system for diagnostic applications. This study reports the results of two ring studies investigating TIL conducted by the International Working Group on Immuno-oncology Biomarkers. The study aim was to determine the intraclass correlation coefficient (ICC) for evaluation of TIL by different pathologists. A total of 120 slides were evaluated by a large group of pathologists with a web-based system in ring study 1 and a more advanced software-system in ring study 2 that included an integrated feedback with standardized reference images. The predefined aim for successful ring studies 1 and 2 was an ICC above 0.7 (lower limit of 95% confidence interval (CI)). In ring study 1 the prespecified endpoint was not reached (ICC: 0.70; 95% CI: 0.62–0.78). On the basis of an analysis of sources of variation, we developed a more advanced digital image evaluation system for ring study 2, which improved the ICC to 0.89 (95% CI: 0.85–0.92). The Fleiss’ kappa value for <60 vs ≥60% TIL improved from 0.45 (ring study 1) to 0.63 in RS2 and the mean concordance improved from 88 to 92%. This large international standardization project shows that reproducible evaluation of TIL is feasible in breast cancer. This opens the way for standardized reporting of tumor immunological parameters in clinical studies and diagnostic practice. The software-guided image evaluation approach used in ring study 2 may be of value as a tool for evaluation of TIL in clinical trials and diagnostic practice. The experience gained from this approach might be applicable to the standardization of other diagnostic parameters in histopathology.Modern Pathology advance online publication, 1 July 2016; doi:10.1038/modpathol.2016.109.

AB - Multiple independent studies have shown that tumor-infiltrating lymphocytes (TIL) are prognostic in breast cancer with potential relevance for response to immune-checkpoint inhibitor therapy. Although many groups are currently evaluating TIL, there is no standardized system for diagnostic applications. This study reports the results of two ring studies investigating TIL conducted by the International Working Group on Immuno-oncology Biomarkers. The study aim was to determine the intraclass correlation coefficient (ICC) for evaluation of TIL by different pathologists. A total of 120 slides were evaluated by a large group of pathologists with a web-based system in ring study 1 and a more advanced software-system in ring study 2 that included an integrated feedback with standardized reference images. The predefined aim for successful ring studies 1 and 2 was an ICC above 0.7 (lower limit of 95% confidence interval (CI)). In ring study 1 the prespecified endpoint was not reached (ICC: 0.70; 95% CI: 0.62–0.78). On the basis of an analysis of sources of variation, we developed a more advanced digital image evaluation system for ring study 2, which improved the ICC to 0.89 (95% CI: 0.85–0.92). The Fleiss’ kappa value for <60 vs ≥60% TIL improved from 0.45 (ring study 1) to 0.63 in RS2 and the mean concordance improved from 88 to 92%. This large international standardization project shows that reproducible evaluation of TIL is feasible in breast cancer. This opens the way for standardized reporting of tumor immunological parameters in clinical studies and diagnostic practice. The software-guided image evaluation approach used in ring study 2 may be of value as a tool for evaluation of TIL in clinical trials and diagnostic practice. The experience gained from this approach might be applicable to the standardization of other diagnostic parameters in histopathology.Modern Pathology advance online publication, 1 July 2016; doi:10.1038/modpathol.2016.109.

UR - http://www.scopus.com/inward/record.url?scp=84976592728&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84976592728&partnerID=8YFLogxK

U2 - 10.1038/modpathol.2016.109

DO - 10.1038/modpathol.2016.109

M3 - Article

JO - Modern Pathology

JF - Modern Pathology

SN - 0893-3952

ER -