Initial Experience with 3D XPoint Main Memory
Jihang Liu
State Key Laboratory of Computer Architecture
Institute of Computing Technology, CAS
University of Chinese Academy of Sciences
liujihang@ict.ac.cn
Shimin Chen*
State Key Laboratory of Computer Architecture
Institute of Computing Technology, CAS
University of Chinese Academy of Sciences
chensm@ict.ac.cn
Abstract—3D XPoint will likely be the first commercially
available main memory NVM solution targeting mainstream
computer systems. Previous database studies on NVM memory
evaluate their proposed techniques mainly on simulated or
emulated NVM hardware. In this paper, we report our initial
experience experimenting with the real 3D XPoint main memory
hardware.
I. INTRODUCTION
A new generation of Non-Volatile Memory (NVM) tech-
nologies, including PCM [1], STT-RAM [2], and Memris-
tor [3], are expected to address the DRAM scaling problem
and become the next generation main memory technology
in future computer systems [4]–[7]. NVM has become a hot
topic in database research in recent years. However, previous
studies evaluate their proposed techniques mainly on simulated
or emulated hardware. In this paper, we report our initial
experience with real NVM main memory hardware.
3D XPoint is an NVM technology jointly developed by
Intel and Micron [8]. Intel first announced the technology
in 2015. 3D XPoint based Optane SSD products are already
available on the market. Intel expects to ship 3D XPoint based
persistent memory DIMM products (a.k.a. Apache Pass, or
AEP) in 2019. As a result, 3D XPoint will likely be the first
commercially available main memory NVM solution targeting
mainstream computer systems.
A number of companies have obtained AEP sample ma-
chines from Intel for testing purposes. We get access to an
AEP machine through a collaboration project with Alibaba.
We are interested in understanding the system architecture of
the AEP machine, and the performance characteristics of the
3D XPoint main memory.
The remainder of the paper is organized as follows. Sec-
tion II describes the interesting features of the AEP ma-
chine. Section III discusses our experimental methodology.
Section IV reports the experimental results. Finally, Section V
concludes the paper.
II. AEP MACHINE
The system architecture of the AEP machine is shown in
Figure 1. We describe its interesting features that are different
from those in existing x86-64 servers.
*Corresponding Author
Fig. 1. AEP machine.
First, there are two CPUs in the machine. From the
/proc/cpuinfo, the CPU model is listed as “Genuine In-
tel(R) CPU 0000%@”. Our guess is that the CPU is a test
model and/or not yet recognized by the Linux kernel. The
CPU supports the new clwb instruction [9]. Given a memory
address, clwb writes back to memory the associated cache
line if it is modified. clwb and sfence can be combined to
persist modified data to the NVM main memory. Compared
to the existing clflush instruction, clwb is expected to be
more efficient because it does not invalidate the associated line
from the CPU cache.
Second, the AEP machine is equipped with two 3D XPoint
main memory modules. Each module is 256GB large, and is
connected to a CPU. For a CPU, one 3D XPoint module is
local, the other module is remote. In other words, there are
NUMA effects for NVM memory accesses.
Third, the machine is running a 4.9.75 version Linux kernel.
Under the /dev directory, there are two special devices that
represent the two 3D XPoint modules. They are named as
/dev/pmemX, where X is a number.
Finally, there are two ways to exploit the 3D XPoint
modules. File systems are installed on the devices using the
fsdax mode. fsdax supports direct accesses to the NVM device
without going through the OS page cache. More interestingly,
we can use PMDK (Persistent Memory Development Kit)
1
to
mmap an NVM file into the virtual address space of the run-
ning process, and then access the NVM memory using normal
load and store instructions, essentially as main memory. We
focus on the latter approach in this paper.
1
http://pmem.io/pmdk/libpmem/
资源评论
