4.20.1深度神经网络提高放射科医生在乳腺癌筛查中的表现资源-CSDN文库

182 浏览量 2024-08-13 12:27:53 上传评论收藏 5.4MB PDF 举报

资源推荐

资源详情

资源评论

1184 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 39, NO. 4, APRIL 2020

Deep Neural Networks Improve Radiologists’

Performance in Breast Cancer Screening

Nan Wu , Jason Phang , Jungkyu Park, Yiqiu Shen , Zhe Huang, Masha Zorin, Stanisław Jastrze˛bski,

Thibault Févry, Joe Katsnelson, Eric Kim

, Stacey Wolfson, Ujas Parikh, Sushma Gaddam,

Leng Leng Young Lin, Kara Ho, Joshua D. Weinstein, Beatriu Reig, Yiming Gao, Hildegard Toth,

Kristine Pysarenko, Alana Lewin, Jiyon Lee, Krystal Airola, Eralda Mema,

Stephanie Chung, Esther Hwang, Naziya Samreen, S. Gene Kim,

Laura Heacock

, Linda Moy, Kyunghyun Cho,

and Krzysztof J. Geras

Abstract

— We present a deep convolutional neural net-

work for breast cancer screening exam classiﬁcation,

trained, and evaluated on over 200 000 exams (over

1 000 000 images). Our network achieves an AUC of 0.895 in

predicting the presence of cancer in the breast, when

tested on the screening population. We attribute the high

accuracy to a few technical advances. 1) Our network’s

Manuscript received June 19, 2019; revised September 16, 2019;

accepted September 28, 2019. Date of publication October 7, 2019;

date of current version April 1, 2020. This work was supported in part by

the National Institutes of Health under Grant R21CA225175 and Grant

P41EB017183.

(Corresponding author: Krzysztof J. Geras.)

N. Wu, J. Phang, J. Park, Y. Shen, Z. Huang, and T. Févry are with

the Center for Data Science, New York University, New York, NY 10011

USA.

M. Zorin was with NYU Courant Institute of Mathematical Sciences,

New York University, New York, NY 10011 USA. She is now with

the Department of Computer Science and Technology, University of

Cambridge, Cambridge CB3 0FD, U.K.

S. Jastrze˛bski is with the Faculty of Mathematics and Information

Technologies, Jagiellonian University, 30-348 Kraków, Poland.

J. Katsnelson, E. Kim, S. Wolfson, U. Parikh, S. Gaddam, L. L. Y. Lin,

J. D. Weinstein, K. Airola, E. Mema, S. Chung, E. Hwang, and N. Samreen

are with the Department of Radiology, School of Medicine, New York

University, New York, NY 10016 USA.

K. Ho is with the SUNY Downstate College of Medicine, New York,

NY 11203 USA.

B. Reig, Y. Gao, H. Toth, K. Pysarenko, A. Lewin, J. Lee, and L. Heacock

are with the Department of Radiology, School of Medicine, New York

University, New York, NY 10016 USA, and also with the Perlmutter

Cancer Center, NYU Langone Health, New York, NY 10016 USA.

S. G. Kim and L. Moy are with the Department of Radiology, School

of Medicine, New York University, New York, NY 10016 USA, with the

Perlmutter Cancer Center, NYU Langone Health, New York, NY 10016

USA, and also with the Center for Advanced Imaging Innovation and

Research, NYU Langone Health, New York, NY 10016 USA.

K. Cho is with the Center for Data Science, New York University, New

York, NY 10011 USA, and also with the Courant Institute of Mathematical

Sciences, New York University, New York, NY 10012 USA.

K. J. Geras is with the Department of Radiology, School of Medicine,

New York University, New York, NY 10016 USA, with the Center for Data

Science, New York University, New York, NY 10011 USA, and also with

Center for Advanced Imaging Innovation and Research, NYU Langone

Health, New York, NY 10016 USA (e-mail: k.j.geras@nyu.edu).

This article has supplementary downloadable material available at

http://ieeexplore.ieee.org, provided by the authors.

Color versions of one or more of the ﬁgures in this article are available

online at http://ieeexplore.ieee.org.

Digital Object Identiﬁer 10.1109/TMI.2019.2945514

novel two-stage architecture and training procedure, which

allows us to use a high-capacity patch-level network to

learn from pixel-level labels alongside a network learning

from macroscopic breast-level labels. 2) A custom ResNet-

based network used as a building block of our model,

whose balance of depth and width is optimized for high-

resolution medical images. 3) Pretraining the network on

screening BI-RADS classiﬁcation, a related task with more

noisy labels. 4) Combining multiple input views in an opti-

mal way among a number of possible choices. To validate

our model, we conducted a reader study with 14 readers,

each reading 720 screening mammogram exams, and show

that our model is as accurate as experienced radiologists

when presented with the same data. We also show that

a hybrid model, averaging the probability of malignancy

predicted by a radiologist with a prediction of our neural

network, is more accurate than either of the two sepa-

rately. To further understand our results, we conduct a

thorough analysis of our network’s performance on dif-

ferent subpopulations of the screening population, the

model’s design, training procedure, errors, and properties

of its internal representations. Our best models are pub-

licly available at https://github.com/nyukat/breast_cancer_

classiﬁer.

Index Terms

— Deep learning, deep convolutional neural

networks, breast cancer screening, mammography.

I. INTRODUCTION

REAST cancer is the second leading cancer-related

cause of death among women in the US. In 2014,

over 39 million screening and diagnostic mammography

exams were performed in the US. It is estimated that in

2015 232,000 women were diagnosed with breast cancer and

approximately 40,000 died from it [1]. Although mammogra-

phy is the only imaging test that has reduced breast cancer

mortality [2]–[4], there has been discussion regarding the

potential harms of screening, including false positive recalls

and associated false positive biopsies. The vast majority of

the 10–15% of women asked to return following an incon-

clusive screening mammogram undergo another mammogram

and/or ultrasound for clariﬁcation. After the additional imaging

exams, many of these ﬁndings are determined as benign and

only 10–20% are recommended to undergo a needle biopsy for

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

et al.

: DEEP NEURAL NETWORKS IMPROVE RADIOLOGISTS’ PERFORMANCE IN BREAST CANCER SCREENING 1185

further work-up. Among these, only 20–40% yield a diagnosis

of cancer [5]. Evidently, there is an unmet need to shift

the balance of routine breast cancer screening towards more

beneﬁt and less harm.

Traditional computer-aided detection (CAD) in mammog-

raphy is routinely used by radiologists to assist with image

interpretation, despite multicenter studies showing these CAD

programs do not improve their diagnostic performance [6].

These programs typically use handcrafted features to mark

sites on a mammogram that appear distinct from normal

tissue. The radiologist decides whether to recall these ﬁndings,

determining clinical signiﬁcance and actionability. Recent

developments in deep learning [7]—in particular, deep convo-

lutional neural networks (CNNs) [8]–[12]—open possibilities

for creating a new generation of CAD-like tools.

This paper makes several technical contributions towards

the goal of developing neural networks to support radiologists

in interpreting breast cancer screening exams. (i) We introduce

a novel two-stage neural network for incorporating global and

local information with an appropriate training procedure. This

allowed us to use a very high-capacity patch-level network

to learn from pixel-level labels alongside a network learning

from macroscopic breast-level labels. With this strategy, our

model not only achieves a human-competitive performance

but also produces interpretable heatmaps indicating locations

of suspicious ﬁndings. Additionally, we show the utility of

pixel-level labels even in a regime where we have a lot

of image-level labels. (ii) We demonstrate the feasibility of

training and evaluating the network with over 1,000,000 high-

resolution mammographic images–an extremely large data set

in medical imaging, not just for breast cancer screening. This

has a signiﬁcant value in both informing future research design

priorities as well as showing a proof-of-concept and proof-of-

value of this approach. We further perform a careful error

analysis of our predictions, and identify patterns that our

network was incapable of capturing, which will inform future

architecture designs. (iii) To use as a building block of our

network, we propose a novel variant of a ResNet speciﬁcally

designed for medical imaging, which has a balance of depth

and width that allows the model to process a very large

image while maintaining reasonable memory consumption.

(iv) We evaluate the utility of pretraining the network using

a related task with a more noisy outcome (screening BI-

RADS classiﬁcation) and ﬁnd it to be a very important part

of the pipeline that markedly improves the performance of

our models. This is of particular signiﬁcance in medical

imaging where most data sets are small. (v) We evaluate

a number of ways to combine information from different

mammographic views within a single neural network. The

results of this analysis are also of value to a broader audience–

including radiologists, particularly pertaining to the margin in

performance between models trained on a subset of the views.

We are not aware of any prior analysis like this, even though it

is common for medical imaging tasks to have multiple inputs.

(vi) We have made the code and weights of our best models

available at https://github.com/nyukat/breast_cancer_classiﬁer.

With this contribution, research groups that are working on

improving screening mammography, who may not have access

Fig. 1. Examples of breast cancer screening exams. First row: both

breasts without any ﬁndings; second row: left breast with no ﬁndings and

right breast with a malignant ﬁnding; third row: left breast with a benign

ﬁnding and right breast with no ﬁndings.

to a large training dataset like ours, will be able to directly use

our model in their research or use our pretrained weights as

an initialization to train models with less data. By making our

models public, we invite other groups to validate our results

and test their robustness to shifts in the data distribution.

II. D

ATA

Our retrospective study was approved by our institutional

review board and was compliant with the Health Insurance

Portability and Accountability Act. Informed consent was

waived. This dataset

is a larger and more carefully curated

version of a dataset used in our earlier work [14], [15].

The dataset includes 229,426 digital screening mammography

exams (1,001,093 images) from 141,473 patients. Each exam

contains at least four images,

corresponding to the four

standard views used in screening mammography: R-CC (right

craniocaudal), L-CC (left craniocaudal), R-MLO (right medi-

olateral oblique) and L-MLO (left mediolateral oblique). The

images in the dataset are coming from four types of scanners:

Mammomat Inspiration (22.81%), Mammomat Novation DR

(12.65%), Lorad Selenia (40.92%) and Selenia Dimensions

(23.62%). A few examples of exams are shown in Fig. 1.

To extract labels indicating whether each breast of the

patient was found to have malignant or benign ﬁndings at

the end of the diagnostic pipeline, we relied on pathology

reports from biopsies. We have 5,832 exams with at least one

biopsy performed within 120 days of the screening mammo-

gram. Among these, biopsies conﬁrmed malignant ﬁndings for

Details of its statistics and how it was extracted can be found in a separate

technical report [13].

Some exams contain more than one image per view as technologists may

need to repeat an image or provide a supplemental view to completely image

the breast in a screening examination.

1186 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 39, NO. 4, APRIL 2020

Fig. 2. An example of a segmentation performed by a radiologist.

Left: the original image. Right: the image with lesions requiring a biopsy

highlighted. The malignant ﬁnding is highlighted with red and benign

ﬁnding with green.

TABLE I

UMBER OF BREASTS WITH MALIGNANT AND BENIGN FINDINGS

BASED ON THE LABELS EXTRACTED FROM THE PATHOLOGY

REPORTS,BROKEN DOWN ACCORDING TO WHETHER

THE

FINDINGS WERE VISIBLE OR OCCULT

985 (8.4%) breasts and benign ﬁndings for 5,556 (47.6%)

breasts. 234 (2.0%) breasts had both malignant and benign

ﬁndings. For the remaining screening exams that were not

matched with a biopsy, we assigned labels corresponding to

the absence of malignant and benign ﬁndings in both breasts.

For all exams matched with biopsies, we asked a group

of radiologists (provided with the corresponding pathology

reports) to retrospectively indicate the location of the biopsied

lesions at a pixel level. An example of such a segmentation

is shown in Fig. 2. We found that approximately 32.8% of

exams were mammographically occult, i.e., the lesions that

were biopsied were not visible on mammography, even retro-

spectively, and were identiﬁed using other imaging modalities:

ultrasound or MRI. See Table I for more details.

III. D

EEP CNNSFORCANCER CLASSIFICATION

As some breasts contain both malignant and benign ﬁnd-

ings, we formulate breast cancer screening classiﬁcation as a

learning task using the multi-task learning framework [16].

That is, for each breast, we assign two binary labels: the

absence/presence of malignant ﬁndings in a breast (denoted by

R,m

and y

L,m

), and the absence/presence of benign ﬁndings in

a breast (denoted by y

R,b

and y

L,b

). With left and right breasts,

each exam has a total of four binary labels. Our goal is to

produce four predictions corresponding to the four labels for

each exam (denoted by ˆy

R,m

, ˆy

L,m

, ˆy

R,b

and ˆy

L,b

). Although

we are primarily interested in accurately predicting presence

Fig. 3. A schematic representation of how we formulated breast cancer

exam classiﬁcation as a learning task. The main task that we intend

the model to learn is malignant/not malignant classiﬁcation. The task of

benign/not benign classiﬁcation is used as an auxiliary task regularizing

the network.

or absence of malignant ﬁndings, predicting the presence or

absence of benign ﬁndings serves an important role of an

auxiliary task regularizing learning the primary task. As input,

we take four high-resolution images corresponding to the four

standard screening mammography views (denoted by x

R−CC

L−CC

, x

R−MLO

and x

L−MLO

). We crop each image to a ﬁxed

size of 2677×1942 pixels for CC views and 2974×1748 pixels

for MLO views.

See Fig. 3 for a schematic representation.

IV. M

ODEL ARCHITECTURE AND TRAINING

We trained deep multi-view CNNs of four different

architectures shown in Fig. 5, inspired by prior work of

Geras et al. [14]. All of these networks consist of two core

modules: (i) four view-speciﬁc columns, each based on the

ResNet architecture [11] that output a ﬁxed-dimension hidden

representation for each mammography view, and (ii) two fully

connected layers to map the computed hidden representations

to the output predictions. The models differ in how the view-

speciﬁc hidden representations from all views are aggregated

to produce the ﬁnal predictions. We considered the following

variants.

1) The ‘view-wise’ model (Fig. 5(a)) concatenates L-CC and

R-CC representations, and L-MLO and R-MLO represen-

tations. It makes separate predictions for CC and MLO

views, which are averaged during inference.

2) The ‘image-wise’ model (Fig. 5(b)) makes a prediction

for each of the four views independently. Corresponding

predictions are averaged during inference.

3) The ‘side-wise’ (Fig. 5(c)) model ﬁrst concatenates L-CC

and L-MLO representations, and R-CC and R-MLO

representations, then makes predictions for each breast

separately.

4) The ‘joint’ model (Fig. 5(d)) concatenates the represen-

tations of all four views and jointly predicts malignant

and benign ﬁndings for both breasts.

In all models, we used four ResNet-based 22-layer networks

(ResNet-22) as columns computing a 256-dimension hidden

The sizes and locations of the cropping window for each image are adjusted

to contain as much of the breast tissue as possible using a method explained

in the section 2.D of the technical report on the dataset [13].

剩余10页未读，继续阅读

评论收藏

内容反馈

托比-马奎尔

粉丝: 1721
资源: 15

4.20.1 深度神经网络提高放射科医生在乳腺癌筛查中的表现

最新资源

4.20.1 深度神经网络提高放射科医生在乳腺癌筛查中的表现

Docker Desktop4.20.1

Driver-R4.20.4201.zip驱动

IAR V4.20.1注册机

stm32cubemx 4.20.1

vdsm-gluster-4.20.39.1-1.el7.x86_64.rpm

中文原厂固件 WRT54G V1-V4.0_4.20.9_SC_code.rar

遥志邮件服务器DBMail企业版 4.20.1无限用户

U盘芯片检测工具2023最新版-(Chip Genius) v4.20.1107

Linux4.20.12内核

U盘芯检测工具(Chip Genius) v4.20.1107.rar

AceMoney 4.20.6

strace4.20.tar.xz

ScadaV4.20.17.rar

Tampermonkey BETA 4.20.6187-0篡改猴测试版 Chrome/Edge插件解压即用

Reflector7.5.4.20.zip

Linux 内核最新稳定版linux-4.20.4.tar.xz

nspr-4.20.tar.gz

GLSLangSpec.4.20.8.clean

memtest86+-4.20.usb.installer

YOLOv8-deepsort 实现智能车辆目标检测+车辆跟踪+车辆计数

Transformer模型实现长期预测并可视化结果（附代码+数据集+原理介绍）

YOLOv8网络结构图，自制visio文件，yolov8.vsds，需要的自取，在原有的基础上直接改就行了

yolov8(2023年8月版本),已经下好yolov8s.pt和yolov8n.pt

社交平台上经济类话题的文章热度信息，数据是真实的，但不是真实日期

行人跌倒数据集（VOC格式）

YOLOV5口罩检测数据集+代码+模型 2000张标注好的数据+教学视频.zip

CIFAR10数据集免费下载

Deep Learning Tuning Playbook（中译版）

Unet眼底血管图像分割数据集+代码+模型+系统界面+教学视频.zip

最新资源