Data Transfer Matters for GPU Computing
Yusuke Fujii
∗
, Takuya Azumi
†
, Nobuhiko Nishio
†
, Shinpei Kato
‡
and Masato Edahiro
‡
∗
Graduate School of Information Science and Engineering, Ritsumeikan University
†
College of Information Science and Engineering, Ritsumeikan University
‡
School of Information Science, Nagoya University
Abstract—Graphics processing units (GPUs) embrace many-
core compute devices where massively parallel compute threads
are offloaded from CPUs. This heterogeneous nature of GPU
computing raises non-trivial data transfer problems especially
against latency-critical real-time systems. However even the basic
characteristics of data transfers associated with GPU computing
are not well studied in the literature. In this paper, we investigate
and characterize currently-achievable data transfer methods of
cutting-edge GPU technology. We implement these methods using
open-source software to compare their performance and latency
for real-world systems. Our experimental results show that the
hardware-assisted direct memory access (DMA) and the I/O read-
and-write access methods are usually the most effective, while
on-chip microcontrollers inside the GPU are useful in terms of
reducing the data transfer latency for concurrent multiple data
streams. We also disclose that CPU priorities can protect the
performance of GPU data transfers.
Keywords-GPGPU; Data Transfer; Latency; Performance; OS
I. INTRODUCTION
Graphics processing units (GPUs) are becoming more and
more commonplace as many-core compute devices. For exam-
ple, NVIDIA GPUs integrate thousands of processing cores
on a single chip and the peak double-precision performance
exceeds 1 TFLOPS while sustaining thermal design power
(TDP) in the same order of magnitude as traditional multi-
core CPUs [1]. This rapid growth of GPUs is due to recent
advances in the programming model, often referred to as
general-purpose computing on GPUs (GPGPU).
Data-parallel and compute-intensive applications receive
significant performance benefits from GPGPU. Currently a
main application of GPGPU is supercomputing [2] but there
are more and more emerging applications in different fields.
Examples include plasma control [3], autonomous driving [4],
software routing [5], encrypted networking [6], and storage
management [7]–[10]. This broad range of applications raises
the need of further developing GPU technology to enhance
scalability of emerging data-parallel and compute-intensive
applications.
GPU programming inevitably incurs data transfers between
the host and the device memory. This resulting latency could
be a performance stopper of I/O bound GPGPU applications.
In fact, the basic performance and latency issues for GPUs are
not well studied in the literature. Given that compute kernels
are offloaded to the GPU, their performance and latency are
more dominated by compiler and hardware technology. How-
ever an optimization of data transfers must be complemented
0.003$
0.03$
0.3$
3$
30$
300$
16$B$
64$B$
256$B$
1$KB$
4$KB$
16$KB$
64$KB$
256$KB$
1$MB$
4$MB$
16$MB$
64$MB$
Data$Transfer$Time$($ms$)
Data$Size$
DMA$
IORW$
(a) Host to Device
0.003$
0.03$
0.3$
3$
30$
300$
1$KB$
4$KB$
16$KB$
64$KB$
256$KB$
1$MB$
4$MB$
16$MB$
64$MB$
Data$Transfer$Time$($ms$)
Data$Size$
DMA$
IORW$
(b) Device to Host
Fig. 1. Performance of DMA and I/O read/write for the NVIDIA GPU.
by system software due to the constraint of PCIe devices [9].
Data transfers may also be interfered by competing workload
on the CPU, while offloaded compute kernels are isolated on
the GPU. These data transfer issues must be well understood
and addressed to build low-latency GPU computing.
The data transfer is particularly an important issue for low-
latency GPU computing. Kato et. al. demonstrated that the data
transfer is a dominant property of GPU-accelerated plasma
control systems [3]. This is a specific application where the
data must be transferred between sensor/actuator devices and
the GPU at a high-rate, but is a good example presenting the
impact of data transfers on GPU computing. Since emerging
applications augmented with GPUs may demand a similar
performance requirement, a better understanding of the GPU
data transfer mechanism is desired.
Figure 1 depicts the average data transfer times of hardware-
based direct memory access (DMA) and memory-mapped I/O
read-and-write access, which are obtained on an NVIDIA
GeForce GTX 480 graphics card using the open-source Linux
driver [9]. Apparently the performance characteristics of the
data transfer are not identical for the host-to-device and device-
to-host directions. In previous work, a very elementary issue of
this performance difference has been discussed [9], but there
is no clear conclusion on what methods can optimize the data
transfer performance, what different methods are available.
Currently we pray that the black-box data transfer mecha-
nism of proprietary software, provided by GPU vendors, is
well optimized to meet the performance that programmers
expect, because hardware details of GPUs are not disclosed to
the public. In order to achieve low-latency GPU computing,
we must understand what latency and performance interference
exist when using the GPU.
To some extent, GPUs are suitable for real-time computing
once workload is offloaded, but host-device data transfers
剩余7页未读，继续阅读
资源评论
