Nuclei counting in microscopy images with three dimensional generative adversarial networks

Shuo Han, Soonam Lee, Chichen Fu, Paul Salama, Kenneth Dunn, Edward J. Delp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Microscopy image analysis can provide substantial information for clinical study and understanding of biological structures. Two-photon microscopy is a type of fluorescence microscopy that can image deep into tissue with near-infrared excitation light. We are interested in methods that can detect and characterize nuclei in 3D fluorescence microscopy image volumes. In general, several challenges exist for counting nuclei in 3D image volumes. These include "crowding" and touching of nuclei, overlapping of nuclei, and shape and size variances of the nuclei. In this paper, a 3D nuclei counter using two different generative adversarial networks (GAN) is proposed and evaluated. Synthetic data that resembles real microscopy image is generated with a GAN and used to train another 3D GAN that counts the number of nuclei. Our approach is evaluated with respect to the number of groundtruth nuclei and compared with common ways of counting used in the biological research. Fluorescence microscopy 3D image volumes of rat kidneys are used to test our 3D nuclei counter. The accuracy results of proposed nuclei counter are compared with the ImageJ's 3D object counter (JACoP) and the 3D watershed. Both the counting accuracy and the object-based evaluation show that the proposed technique is successful for counting nuclei in 3D.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2019
Subtitle of host publicationImage Processing
EditorsBennett A. Landman, Elsa D. Angelini, Elsa D. Angelini, Elsa D. Angelini
PublisherSPIE
ISBN (Electronic)9781510625457
DOIs
StatePublished - Jan 1 2019
EventMedical Imaging 2019: Image Processing - San Diego, United States
Duration: Feb 19 2019Feb 21 2019

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume10949
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2019: Image Processing
CountryUnited States
CitySan Diego
Period2/19/192/21/19

Fingerprint

Three-Dimensional Imaging
Fluorescence microscopy
Fluorescence Microscopy
Microscopy
Microscopic examination
counting
microscopy
nuclei
Crowding
Watersheds
Photons
Image analysis
counters
Rats
Tissue
Infrared radiation
Kidney
Light
fluorescence
Research

Keywords

  • fluorescence microscopy
  • generative adversarial networks
  • nuclei counting
  • synthetic data generation

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Cite this

Han, S., Lee, S., Fu, C., Salama, P., Dunn, K., & Delp, E. J. (2019). Nuclei counting in microscopy images with three dimensional generative adversarial networks. In B. A. Landman, E. D. Angelini, E. D. Angelini, & E. D. Angelini (Eds.), Medical Imaging 2019: Image Processing [109492Y] (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10949). SPIE. https://doi.org/10.1117/12.2512591

Nuclei counting in microscopy images with three dimensional generative adversarial networks. / Han, Shuo; Lee, Soonam; Fu, Chichen; Salama, Paul; Dunn, Kenneth; Delp, Edward J.

Medical Imaging 2019: Image Processing. ed. / Bennett A. Landman; Elsa D. Angelini; Elsa D. Angelini; Elsa D. Angelini. SPIE, 2019. 109492Y (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10949).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Han, S, Lee, S, Fu, C, Salama, P, Dunn, K & Delp, EJ 2019, Nuclei counting in microscopy images with three dimensional generative adversarial networks. in BA Landman, ED Angelini, ED Angelini & ED Angelini (eds), Medical Imaging 2019: Image Processing., 109492Y, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 10949, SPIE, Medical Imaging 2019: Image Processing, San Diego, United States, 2/19/19. https://doi.org/10.1117/12.2512591
Han S, Lee S, Fu C, Salama P, Dunn K, Delp EJ. Nuclei counting in microscopy images with three dimensional generative adversarial networks. In Landman BA, Angelini ED, Angelini ED, Angelini ED, editors, Medical Imaging 2019: Image Processing. SPIE. 2019. 109492Y. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE). https://doi.org/10.1117/12.2512591
Han, Shuo ; Lee, Soonam ; Fu, Chichen ; Salama, Paul ; Dunn, Kenneth ; Delp, Edward J. / Nuclei counting in microscopy images with three dimensional generative adversarial networks. Medical Imaging 2019: Image Processing. editor / Bennett A. Landman ; Elsa D. Angelini ; Elsa D. Angelini ; Elsa D. Angelini. SPIE, 2019. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE).
@inproceedings{c364018c7e5d47a99e2236a87d0e13c5,
title = "Nuclei counting in microscopy images with three dimensional generative adversarial networks",
abstract = "Microscopy image analysis can provide substantial information for clinical study and understanding of biological structures. Two-photon microscopy is a type of fluorescence microscopy that can image deep into tissue with near-infrared excitation light. We are interested in methods that can detect and characterize nuclei in 3D fluorescence microscopy image volumes. In general, several challenges exist for counting nuclei in 3D image volumes. These include {"}crowding{"} and touching of nuclei, overlapping of nuclei, and shape and size variances of the nuclei. In this paper, a 3D nuclei counter using two different generative adversarial networks (GAN) is proposed and evaluated. Synthetic data that resembles real microscopy image is generated with a GAN and used to train another 3D GAN that counts the number of nuclei. Our approach is evaluated with respect to the number of groundtruth nuclei and compared with common ways of counting used in the biological research. Fluorescence microscopy 3D image volumes of rat kidneys are used to test our 3D nuclei counter. The accuracy results of proposed nuclei counter are compared with the ImageJ's 3D object counter (JACoP) and the 3D watershed. Both the counting accuracy and the object-based evaluation show that the proposed technique is successful for counting nuclei in 3D.",
keywords = "fluorescence microscopy, generative adversarial networks, nuclei counting, synthetic data generation",
author = "Shuo Han and Soonam Lee and Chichen Fu and Paul Salama and Kenneth Dunn and Delp, {Edward J.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1117/12.2512591",
language = "English (US)",
series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",
publisher = "SPIE",
editor = "Landman, {Bennett A.} and Angelini, {Elsa D.} and Angelini, {Elsa D.} and Angelini, {Elsa D.}",
booktitle = "Medical Imaging 2019",

}

TY - GEN

T1 - Nuclei counting in microscopy images with three dimensional generative adversarial networks

AU - Han, Shuo

AU - Lee, Soonam

AU - Fu, Chichen

AU - Salama, Paul

AU - Dunn, Kenneth

AU - Delp, Edward J.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Microscopy image analysis can provide substantial information for clinical study and understanding of biological structures. Two-photon microscopy is a type of fluorescence microscopy that can image deep into tissue with near-infrared excitation light. We are interested in methods that can detect and characterize nuclei in 3D fluorescence microscopy image volumes. In general, several challenges exist for counting nuclei in 3D image volumes. These include "crowding" and touching of nuclei, overlapping of nuclei, and shape and size variances of the nuclei. In this paper, a 3D nuclei counter using two different generative adversarial networks (GAN) is proposed and evaluated. Synthetic data that resembles real microscopy image is generated with a GAN and used to train another 3D GAN that counts the number of nuclei. Our approach is evaluated with respect to the number of groundtruth nuclei and compared with common ways of counting used in the biological research. Fluorescence microscopy 3D image volumes of rat kidneys are used to test our 3D nuclei counter. The accuracy results of proposed nuclei counter are compared with the ImageJ's 3D object counter (JACoP) and the 3D watershed. Both the counting accuracy and the object-based evaluation show that the proposed technique is successful for counting nuclei in 3D.

AB - Microscopy image analysis can provide substantial information for clinical study and understanding of biological structures. Two-photon microscopy is a type of fluorescence microscopy that can image deep into tissue with near-infrared excitation light. We are interested in methods that can detect and characterize nuclei in 3D fluorescence microscopy image volumes. In general, several challenges exist for counting nuclei in 3D image volumes. These include "crowding" and touching of nuclei, overlapping of nuclei, and shape and size variances of the nuclei. In this paper, a 3D nuclei counter using two different generative adversarial networks (GAN) is proposed and evaluated. Synthetic data that resembles real microscopy image is generated with a GAN and used to train another 3D GAN that counts the number of nuclei. Our approach is evaluated with respect to the number of groundtruth nuclei and compared with common ways of counting used in the biological research. Fluorescence microscopy 3D image volumes of rat kidneys are used to test our 3D nuclei counter. The accuracy results of proposed nuclei counter are compared with the ImageJ's 3D object counter (JACoP) and the 3D watershed. Both the counting accuracy and the object-based evaluation show that the proposed technique is successful for counting nuclei in 3D.

KW - fluorescence microscopy

KW - generative adversarial networks

KW - nuclei counting

KW - synthetic data generation

UR - http://www.scopus.com/inward/record.url?scp=85068333989&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068333989&partnerID=8YFLogxK

U2 - 10.1117/12.2512591

DO - 10.1117/12.2512591

M3 - Conference contribution

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Medical Imaging 2019

A2 - Landman, Bennett A.

A2 - Angelini, Elsa D.

A2 - Angelini, Elsa D.

A2 - Angelini, Elsa D.

PB - SPIE

ER -