AMD CPU 性能调优知道文档

1

Best Practice Guide - AMD EPYC

Xu Guo, EPCC, UK

Ole Widar Saastad (Editor), University of Oslo, Norway

Version 2.0 by 18-02-2019

Best Practice Guide - AMD EPYC

2

Table of Contents

1. Introduction .............................................................................................................................. 3

2. System Architecture / Configuration ............................................................................................. 4

3.1.2. Compiler Performance ......................................................................................... 11

3.2. Available (Optimized) Numerical Libraries ........................................................................ 13

3.2.1. Performance of libraries ....................................................................................... 13

3.2.2. Examples of numerical library usage ...................................................................... 15

3.3. Available MPI Implementations ....................................................................................... 17

3.4. OpenMP ...................................................................................................................... 18

3.4.1. Compiler Flags ................................................................................................... 18

3.5. Basic Porting Examples .................................................................................................. 18

3.5.1. OpenSBLI ......................................................................................................... 18

3.5.2. CASTEP ........................................................................................................... 19

4.2. General Hints for Interpreting Results from all tools ............................................................ 22

5. Tuning ................................................................................................................................... 24

5.1. Advanced / Aggressive Compiler Flags ............................................................................. 24

5.1.1. GNU compiler ................................................................................................... 24

5.1.2. Intel compiler .................................................................................................... 24

5.1.3. PGI (Portland) compiler ....................................................................................... 24

5.1.4. Compilers and flags ............................................................................................ 24

5.2. Single Core Optimization ............................................................................................... 25

5.2.1. Replace libm library ............................................................................................ 25

5.3. Advanced OpenMP Usage .............................................................................................. 26

5.4.4. Monitoring NUMA pages ..................................................................................... 31

5.5. Possible Kernel Parameter Tuning .................................................................................... 32

5.5.1. NUMA control ................................................................................................... 32

5.5.2. Scheduling control .............................................................................................. 33

6. Debugging .............................................................................................................................. 35

6.1. Available Debuggers ...................................................................................................... 35

6.2. Compiler Flags ............................................................................................................. 35

Further documentation ................................................................................................................. 36

Best Practice Guide - AMD EPYC

3

1. Introduction

Figure 1. The AMD EPYC Processor chip

version 3.0 lanes.

This guide provides information about how to use the AMD EPYC processors in an HPC environment and it

describes some experiences with the use of some common tools for this processor, in addition to a general overview

of the architecture and memory system. Being a NUMA type architecture information about the nature of the

NUMA is relevant. In addition some tuning and optimization techniques as well as debugging are covered also.

In this mini guide we cover the following tools: Compilers, performance libraries, threading libraries (OpenMP),

Message passing libraries (MPI), memory access and allocation libraries, debuggers, performance profilers, etc.

Some benchmarks, in which we compare compilers and libraries have been performed and some recommendations

and hints about how to use the Intel tools with this processor are presented. While the AMD EPYC is a x86-64

architecture it's not fully compatible with Intel processors when it comes to the new features found on the latest

Best Practice Guide - AMD EPYC

4

2. System Architecture / Configuration

SoC. Simultaneous Multithreading (SMT) is supported on the Zen core, which allows each core to run two threads

giving at maximum 64 threads per CPU in total. Each EPYC processor provides 8 memory channels and 128

PCIe 3.0 lanes. EPYC supports both 1-socket and 2-sockets models. In the multi-processor configuration, half

of the PCIe lanes from each processor are used for the communications between the two CPUs through AMD’s

socket-to-socket interconnect, Infinity Fabric [3][4][5][7].

There are several resources available for information about the Zen architecture and the EPYC processor. The

wikichip web site generally is a good source of information [6]. The figures and much of the information below

is taken from the web pages at wikichip and their article about Zen. Detailed information about cache sizes,

pipelining, TLB etc is found there. The table below just lists the cache sizes as they might be of use for many

programmers.

back policy, 4-5 cycles latency for Int, 7-8 cycles latency for FP

Best Practice Guide - AMD EPYC

5

Figure 2. Zen Block diagram

The Zen core contains a battery of different units, it is not a simple task to figure out how two threads are scheduled

fused multiply-add and two units for additions. Combined they can perform 256 bits wide AVX2 instructions.

The chip is optimized for 128 bits operations. The simple integer vector operations (e.g. shift, add) can all be

done in one cycle, half the latency of AMD's previous architecture. Basic floating point math has a latency of

three cycles including multiplication (one additional cycle for double precision). Fused multiply-add (FMA) has

a latency of five cycles.

AMD claim that theoretical floating point performance can be calculated as: Double Precision theoretical Floating

Point performance = #real_cores*8DP flop/clk * core frequency. For a 2 socket system = 2*32cores*8DP flops/

clk * 2.2GHz = 1126.4 Gflops. This includes counting FMA as two flops.