没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
![exe](https://img-home.csdnimg.cn/images/20210720083343.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![application/x-rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![gz](https://img-home.csdnimg.cn/images/20210720083447.png)
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![vsdx](https://img-home.csdnimg.cn/images/20210720083646.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![csv](https://img-home.csdnimg.cn/images/20210720083646.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![](https://csdnimg.cn/release/download_crawler_static/89634359/bg1.jpg)
1184 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 39, NO. 4, APRIL 2020
Deep Neural Networks Improve Radiologists’
Performance in Breast Cancer Screening
Nan Wu , Jason Phang , Jungkyu Park, Yiqiu Shen , Zhe Huang, Masha Zorin, Stanisław Jastrze˛bski,
Thibault Févry, Joe Katsnelson, Eric Kim
, Stacey Wolfson, Ujas Parikh, Sushma Gaddam,
Leng Leng Young Lin, Kara Ho, Joshua D. Weinstein, Beatriu Reig, Yiming Gao, Hildegard Toth,
Kristine Pysarenko, Alana Lewin, Jiyon Lee, Krystal Airola, Eralda Mema,
Stephanie Chung, Esther Hwang, Naziya Samreen, S. Gene Kim,
Laura Heacock
, Linda Moy, Kyunghyun Cho,
and Krzysztof J. Geras
Abstract
— We present a deep convolutional neural net-
work for breast cancer screening exam classification,
trained, and evaluated on over 200 000 exams (over
1 000 000 images). Our network achieves an AUC of 0.895 in
predicting the presence of cancer in the breast, when
tested on the screening population. We attribute the high
accuracy to a few technical advances. 1) Our network’s
Manuscript received June 19, 2019; revised September 16, 2019;
accepted September 28, 2019. Date of publication October 7, 2019;
date of current version April 1, 2020. This work was supported in part by
the National Institutes of Health under Grant R21CA225175 and Grant
P41EB017183.
(Corresponding author: Krzysztof J. Geras.)
N. Wu, J. Phang, J. Park, Y. Shen, Z. Huang, and T. Févry are with
the Center for Data Science, New York University, New York, NY 10011
USA.
M. Zorin was with NYU Courant Institute of Mathematical Sciences,
New York University, New York, NY 10011 USA. She is now with
the Department of Computer Science and Technology, University of
Cambridge, Cambridge CB3 0FD, U.K.
S. Jastrze˛bski is with the Faculty of Mathematics and Information
Technologies, Jagiellonian University, 30-348 Kraków, Poland.
J. Katsnelson, E. Kim, S. Wolfson, U. Parikh, S. Gaddam, L. L. Y. Lin,
J. D. Weinstein, K. Airola, E. Mema, S. Chung, E. Hwang, and N. Samreen
are with the Department of Radiology, School of Medicine, New York
University, New York, NY 10016 USA.
K. Ho is with the SUNY Downstate College of Medicine, New York,
NY 11203 USA.
B. Reig, Y. Gao, H. Toth, K. Pysarenko, A. Lewin, J. Lee, and L. Heacock
are with the Department of Radiology, School of Medicine, New York
University, New York, NY 10016 USA, and also with the Perlmutter
Cancer Center, NYU Langone Health, New York, NY 10016 USA.
S. G. Kim and L. Moy are with the Department of Radiology, School
of Medicine, New York University, New York, NY 10016 USA, with the
Perlmutter Cancer Center, NYU Langone Health, New York, NY 10016
USA, and also with the Center for Advanced Imaging Innovation and
Research, NYU Langone Health, New York, NY 10016 USA.
K. Cho is with the Center for Data Science, New York University, New
York, NY 10011 USA, and also with the Courant Institute of Mathematical
Sciences, New York University, New York, NY 10012 USA.
K. J. Geras is with the Department of Radiology, School of Medicine,
New York University, New York, NY 10016 USA, with the Center for Data
Science, New York University, New York, NY 10011 USA, and also with
Center for Advanced Imaging Innovation and Research, NYU Langone
Health, New York, NY 10016 USA (e-mail: k.j.geras@nyu.edu).
This article has supplementary downloadable material available at
http://ieeexplore.ieee.org, provided by the authors.
Color versions of one or more of the figures in this article are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TMI.2019.2945514
novel two-stage architecture and training procedure, which
allows us to use a high-capacity patch-level network to
learn from pixel-level labels alongside a network learning
from macroscopic breast-level labels. 2) A custom ResNet-
based network used as a building block of our model,
whose balance of depth and width is optimized for high-
resolution medical images. 3) Pretraining the network on
screening BI-RADS classification, a related task with more
noisy labels. 4) Combining multiple input views in an opti-
mal way among a number of possible choices. To validate
our model, we conducted a reader study with 14 readers,
each reading 720 screening mammogram exams, and show
that our model is as accurate as experienced radiologists
when presented with the same data. We also show that
a hybrid model, averaging the probability of malignancy
predicted by a radiologist with a prediction of our neural
network, is more accurate than either of the two sepa-
rately. To further understand our results, we conduct a
thorough analysis of our network’s performance on dif-
ferent subpopulations of the screening population, the
model’s design, training procedure, errors, and properties
of its internal representations. Our best models are pub-
licly available at https://github.com/nyukat/breast_cancer_
classifier.
Index Terms
— Deep learning, deep convolutional neural
networks, breast cancer screening, mammography.
I. INTRODUCTION
B
REAST cancer is the second leading cancer-related
cause of death among women in the US. In 2014,
over 39 million screening and diagnostic mammography
exams were performed in the US. It is estimated that in
2015 232,000 women were diagnosed with breast cancer and
approximately 40,000 died from it [1]. Although mammogra-
phy is the only imaging test that has reduced breast cancer
mortality [2]–[4], there has been discussion regarding the
potential harms of screening, including false positive recalls
and associated false positive biopsies. The vast majority of
the 10–15% of women asked to return following an incon-
clusive screening mammogram undergo another mammogram
and/or ultrasound for clarification. After the additional imaging
exams, many of these findings are determined as benign and
only 10–20% are recommended to undergo a needle biopsy for
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
![](https://csdnimg.cn/release/download_crawler_static/89634359/bg2.jpg)
WU
et al.
: DEEP NEURAL NETWORKS IMPROVE RADIOLOGISTS’ PERFORMANCE IN BREAST CANCER SCREENING 1185
further work-up. Among these, only 20–40% yield a diagnosis
of cancer [5]. Evidently, there is an unmet need to shift
the balance of routine breast cancer screening towards more
benefit and less harm.
Traditional computer-aided detection (CAD) in mammog-
raphy is routinely used by radiologists to assist with image
interpretation, despite multicenter studies showing these CAD
programs do not improve their diagnostic performance [6].
These programs typically use handcrafted features to mark
sites on a mammogram that appear distinct from normal
tissue. The radiologist decides whether to recall these findings,
determining clinical significance and actionability. Recent
developments in deep learning [7]—in particular, deep convo-
lutional neural networks (CNNs) [8]–[12]—open possibilities
for creating a new generation of CAD-like tools.
This paper makes several technical contributions towards
the goal of developing neural networks to support radiologists
in interpreting breast cancer screening exams. (i) We introduce
a novel two-stage neural network for incorporating global and
local information with an appropriate training procedure. This
allowed us to use a very high-capacity patch-level network
to learn from pixel-level labels alongside a network learning
from macroscopic breast-level labels. With this strategy, our
model not only achieves a human-competitive performance
but also produces interpretable heatmaps indicating locations
of suspicious findings. Additionally, we show the utility of
pixel-level labels even in a regime where we have a lot
of image-level labels. (ii) We demonstrate the feasibility of
training and evaluating the network with over 1,000,000 high-
resolution mammographic images–an extremely large data set
in medical imaging, not just for breast cancer screening. This
has a significant value in both informing future research design
priorities as well as showing a proof-of-concept and proof-of-
value of this approach. We further perform a careful error
analysis of our predictions, and identify patterns that our
network was incapable of capturing, which will inform future
architecture designs. (iii) To use as a building block of our
network, we propose a novel variant of a ResNet specifically
designed for medical imaging, which has a balance of depth
and width that allows the model to process a very large
image while maintaining reasonable memory consumption.
(iv) We evaluate the utility of pretraining the network using
a related task with a more noisy outcome (screening BI-
RADS classification) and find it to be a very important part
of the pipeline that markedly improves the performance of
our models. This is of particular significance in medical
imaging where most data sets are small. (v) We evaluate
a number of ways to combine information from different
mammographic views within a single neural network. The
results of this analysis are also of value to a broader audience–
including radiologists, particularly pertaining to the margin in
performance between models trained on a subset of the views.
We are not aware of any prior analysis like this, even though it
is common for medical imaging tasks to have multiple inputs.
(vi) We have made the code and weights of our best models
available at https://github.com/nyukat/breast_cancer_classifier.
With this contribution, research groups that are working on
improving screening mammography, who may not have access
Fig. 1. Examples of breast cancer screening exams. First row: both
breasts without any findings; second row: left breast with no findings and
right breast with a malignant finding; third row: left breast with a benign
finding and right breast with no findings.
to a large training dataset like ours, will be able to directly use
our model in their research or use our pretrained weights as
an initialization to train models with less data. By making our
models public, we invite other groups to validate our results
and test their robustness to shifts in the data distribution.
II. D
ATA
Our retrospective study was approved by our institutional
review board and was compliant with the Health Insurance
Portability and Accountability Act. Informed consent was
waived. This dataset
1
is a larger and more carefully curated
version of a dataset used in our earlier work [14], [15].
The dataset includes 229,426 digital screening mammography
exams (1,001,093 images) from 141,473 patients. Each exam
contains at least four images,
2
corresponding to the four
standard views used in screening mammography: R-CC (right
craniocaudal), L-CC (left craniocaudal), R-MLO (right medi-
olateral oblique) and L-MLO (left mediolateral oblique). The
images in the dataset are coming from four types of scanners:
Mammomat Inspiration (22.81%), Mammomat Novation DR
(12.65%), Lorad Selenia (40.92%) and Selenia Dimensions
(23.62%). A few examples of exams are shown in Fig. 1.
To extract labels indicating whether each breast of the
patient was found to have malignant or benign findings at
the end of the diagnostic pipeline, we relied on pathology
reports from biopsies. We have 5,832 exams with at least one
biopsy performed within 120 days of the screening mammo-
gram. Among these, biopsies confirmed malignant findings for
1
Details of its statistics and how it was extracted can be found in a separate
technical report [13].
2
Some exams contain more than one image per view as technologists may
need to repeat an image or provide a supplemental view to completely image
the breast in a screening examination.
![](https://csdnimg.cn/release/download_crawler_static/89634359/bg3.jpg)
1186 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 39, NO. 4, APRIL 2020
Fig. 2. An example of a segmentation performed by a radiologist.
Left: the original image. Right: the image with lesions requiring a biopsy
highlighted. The malignant finding is highlighted with red and benign
finding with green.
TABLE I
N
UMBER OF BREASTS WITH MALIGNANT AND BENIGN FINDINGS
BASED ON THE LABELS EXTRACTED FROM THE PATHOLOGY
REPORTS,BROKEN DOWN ACCORDING TO WHETHER
THE
FINDINGS WERE VISIBLE OR OCCULT
985 (8.4%) breasts and benign findings for 5,556 (47.6%)
breasts. 234 (2.0%) breasts had both malignant and benign
findings. For the remaining screening exams that were not
matched with a biopsy, we assigned labels corresponding to
the absence of malignant and benign findings in both breasts.
For all exams matched with biopsies, we asked a group
of radiologists (provided with the corresponding pathology
reports) to retrospectively indicate the location of the biopsied
lesions at a pixel level. An example of such a segmentation
is shown in Fig. 2. We found that approximately 32.8% of
exams were mammographically occult, i.e., the lesions that
were biopsied were not visible on mammography, even retro-
spectively, and were identified using other imaging modalities:
ultrasound or MRI. See Table I for more details.
III. D
EEP CNNSFORCANCER CLASSIFICATION
As some breasts contain both malignant and benign find-
ings, we formulate breast cancer screening classification as a
learning task using the multi-task learning framework [16].
That is, for each breast, we assign two binary labels: the
absence/presence of malignant findings in a breast (denoted by
y
R,m
and y
L,m
), and the absence/presence of benign findings in
a breast (denoted by y
R,b
and y
L,b
). With left and right breasts,
each exam has a total of four binary labels. Our goal is to
produce four predictions corresponding to the four labels for
each exam (denoted by ˆy
R,m
, ˆy
L,m
, ˆy
R,b
and ˆy
L,b
). Although
we are primarily interested in accurately predicting presence
Fig. 3. A schematic representation of how we formulated breast cancer
exam classification as a learning task. The main task that we intend
the model to learn is malignant/not malignant classification. The task of
benign/not benign classification is used as an auxiliary task regularizing
the network.
or absence of malignant findings, predicting the presence or
absence of benign findings serves an important role of an
auxiliary task regularizing learning the primary task. As input,
we take four high-resolution images corresponding to the four
standard screening mammography views (denoted by x
R−CC
,
x
L−CC
, x
R−MLO
and x
L−MLO
). We crop each image to a fixed
size of 2677×1942 pixels for CC views and 2974×1748 pixels
for MLO views.
3
See Fig. 3 for a schematic representation.
IV. M
ODEL ARCHITECTURE AND TRAINING
We trained deep multi-view CNNs of four different
architectures shown in Fig. 5, inspired by prior work of
Geras et al. [14]. All of these networks consist of two core
modules: (i) four view-specific columns, each based on the
ResNet architecture [11] that output a fixed-dimension hidden
representation for each mammography view, and (ii) two fully
connected layers to map the computed hidden representations
to the output predictions. The models differ in how the view-
specific hidden representations from all views are aggregated
to produce the final predictions. We considered the following
variants.
1) The ‘view-wise’ model (Fig. 5(a)) concatenates L-CC and
R-CC representations, and L-MLO and R-MLO represen-
tations. It makes separate predictions for CC and MLO
views, which are averaged during inference.
2) The ‘image-wise’ model (Fig. 5(b)) makes a prediction
for each of the four views independently. Corresponding
predictions are averaged during inference.
3) The ‘side-wise’ (Fig. 5(c)) model first concatenates L-CC
and L-MLO representations, and R-CC and R-MLO
representations, then makes predictions for each breast
separately.
4) The ‘joint’ model (Fig. 5(d)) concatenates the represen-
tations of all four views and jointly predicts malignant
and benign findings for both breasts.
In all models, we used four ResNet-based 22-layer networks
(ResNet-22) as columns computing a 256-dimension hidden
3
The sizes and locations of the cropping window for each image are adjusted
to contain as much of the breast tissue as possible using a method explained
in the section 2.D of the technical report on the dataset [13].
剩余10页未读,继续阅读
资源评论
![avatar-default](https://csdnimg.cn/release/downloadcmsfe/public/img/lazyLogo2.1882d7f4.png)
![avatar](https://profile-avatar.csdnimg.cn/24a96a15ff9b43b995063c35763f0dd3_qq_47896523.jpg!1)
托比-马奎尔
- 粉丝: 1721
- 资源: 15
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![voice](https://csdnimg.cn/release/downloadcmsfe/public/img/voice.245cc511.png)
![center-task](https://csdnimg.cn/release/downloadcmsfe/public/img/center-task.c2eda91a.png)
最新资源
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![feedback](https://img-home.csdnimg.cn/images/20220527035711.png)
![feedback](https://img-home.csdnimg.cn/images/20220527035711.png)
![feedback-tip](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![dialog-icon](https://csdnimg.cn/release/downloadcmsfe/public/img/green-success.6a4acb44.png)