DU-06920-001 _v12.0 through 12.1 | February 2021
Virtual GPU Software
User Guide
Virtual GPU Software DU-06920-001 _v12.0 through 12.1|ii
Table of Contents
Chapter1.Introduction to NVIDIA vGPU Software..............................................................1
1.1.How NVIDIA vGPU Software Is Used....................................................................................... 1
1.1.2.GPU Pass-Through.............................................................................................................1
1.1.3.Bare-Metal Deployment.....................................................................................................1
1.2.NVIDIA vGPU Software Features............................................................................................. 2
1.2.1.GPU Instance Support on NVIDIA vGPU Software............................................................2
1.2.2.API Support on NVIDIA vGPU............................................................................................ 3
1.2.3.NVIDIA CUDA Toolkit and OpenCL Support on NVIDIA vGPU Software...........................4
1.2.4.Additional vWS Features....................................................................................................5
1.2.5.NVIDIA GPU Cloud (NGC) Containers Support on NVIDIA vGPU Software...................... 5
1.2.6.NVIDIA GPU Operator Support.......................................................................................... 6
1.3.How this Guide Is Organized....................................................................................................6
Chapter2.Installing and Configuring NVIDIA Virtual GPU Manager..................................8
2.1.About NVIDIA Virtual GPUs...................................................................................................... 8
2.1.1.NVIDIA vGPU Architecture................................................................................................. 8
2.1.1.1.Time-Sliced NVIDIA vGPU Internal Architecture....................................................... 9
2.1.1.2.MIG-Backed NVIDIA vGPU Internal Architecture.....................................................10
2.1.2.About Virtual GPU Types..................................................................................................11
2.1.3.Virtual Display Resolutions for Q-series and B-series vGPUs.......................................13
2.1.4.Valid Virtual GPU Configurations on a Single GPU........................................................ 14
2.1.4.1.Valid Time-Sliced Virtual GPU Configurations on a Single GPU............................. 14
2.1.4.2.Valid MIG-Backed Virtual GPU Configurations on a Single GPU.............................15
2.1.5.Guest VM Support............................................................................................................ 15
2.1.5.1.Windows Guest VM Support......................................................................................16
2.1.5.2.Linux Guest VM support............................................................................................16
2.2.Prerequisites for Using NVIDIA vGPU................................................................................... 16
2.3.Switching the Mode an NVIDIA A40 or NVIDIA RTX A6000 GPU........................................... 17
2.4.Switching the Mode of a Tesla M60 or M6 GPU....................................................................17
2.5.Installing and Configuring the NVIDIA Virtual GPU Manager for Citrix Hypervisor..............18
2.5.1.Installing and Updating the NVIDIA Virtual GPU Manager for Citrix Hypervisor........... 18
2.5.1.1.Installing the RPM package for Citrix Hypervisor....................................................18
2.5.1.2.Updating the RPM Package for Citrix Hypervisor....................................................19
2.5.1.3.Installing or Updating the Supplemental Pack for Citrix Hypervisor...................... 19
2.5.1.4. Verifying the Installation of the NVIDIA vGPU Software for Citrix Hypervisor
Package.............................................................................................................................. 21
Virtual GPU Software DU-06920-001 _v12.0 through 12.1|iii
2.5.2.Configuring a Citrix Hypervisor VM with Virtual GPU.....................................................22
2.6.Installing the Virtual GPU Manager Package for Linux KVM................................................23
2.7.Installing and Configuring the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux
KVM or RHV............................................................................................................................... 24
2.7.1.Installing the NVIDIA Virtual GPU Manager for Red Hat Enterprise Linux KVM or
RHV......................................................................................................................................... 25
2.7.1.1.Installing the Virtual GPU Manager Package for Red Hat Enterprise Linux KVM
or RHV.................................................................................................................................25
2.7.1.2.Verifying the Installation of the NVIDIA vGPU Software for Red Hat Enterprise
Linux KVM or RHV............................................................................................................. 26
2.7.2.Adding a vGPU to a Red Hat Virtualization (RHV) VM.....................................................27
2.7.3.Getting the BDF and Domain of a GPU on Red Hat Enterprise Linux KVM................... 28
2.7.4.Creating an NVIDIA vGPU on Red Hat Enterprise Linux KVM........................................29
2.7.5.Adding One or More vGPUs to a Red Hat Enterprise Linux KVM VM.............................31
2.7.5.1.Adding One or More vGPUs to a Red Hat Enterprise Linux KVM VM by Using
virsh.....................................................................................................................................32
2.7.5.2.Adding One or More vGPUs to a Red Hat Enterprise Linux KVM VM by Using the
QEMU Command Line....................................................................................................... 33
2.7.6.Setting vGPU Plugin Parameters on Red Hat Enterprise Linux KVM............................33
2.7.7.Deleting a vGPU on Red Hat Enterprise Linux KVM.......................................................34
2.7.8.Preparing a GPU Configured for Pass-Through for Use with vGPU..............................35
2.7.9.NVIDIA vGPU Information in the sysfs File System........................................................36
2.8.Installing and Configuring the NVIDIA Virtual GPU Manager for VMware vSphere............. 39
2.8.1.Installing and Updating the NVIDIA Virtual GPU Manager for vSphere.........................40
2.8.1.1.Installing the NVIDIA Virtual GPU Manager Package for vSphere.......................... 41
2.8.1.2.Updating the NVIDIA Virtual GPU Manager Package for vSphere...........................41
2.8.1.3.Verifying the Installation of the NVIDIA vGPU Software Package for vSphere...... 42
2.8.2.Configuring VMware vMotion with vGPU for VMware vSphere...................................... 43
2.8.3.Changing the Default Graphics Type in VMware vSphere 6.5 and Later....................... 44
2.8.4.Configuring a vSphere VM with NVIDIA vGPU................................................................ 48
2.8.5.Configuring a vSphere VM with VMware vSGA............................................................... 51
2.9.Configuring a GPU for MIG-Backed vGPUs...........................................................................52
2.10.Disabling MIG Mode for One or More GPUs........................................................................55
2.11.Disabling and Enabling ECC Memory..................................................................................56
2.11.1.Disabling ECC Memory.................................................................................................. 57
2.11.2.Enabling ECC Memory................................................................................................... 58
Chapter3.Using GPU Pass-Through................................................................................ 60
3.1.Display Resolutions for Physical GPUs................................................................................. 61
3.2.Using GPU Pass-Through on Citrix Hypervisor.....................................................................62
Virtual GPU Software DU-06920-001 _v12.0 through 12.1|iv
3.2.1.Configuring a VM for GPU Pass Through by Using XenCenter...................................... 62
3.2.2.Configuring a VM for GPU Pass Through by Using xe................................................... 63
3.3.Using GPU Pass-Through on Red Hat Enterprise Linux KVM..............................................64
3.3.1.Configuring a VM for GPU Pass-Through by Using Virtual Machine Manager (virt-
manager)................................................................................................................................ 64
3.3.2.Configuring a VM for GPU Pass-Through by Using virsh...............................................65
3.3.3.Configuring a VM for GPU Pass-Through by Using the QEMU Command Line.............66
3.3.4.Preparing a GPU Configured for vGPU for Use in Pass-Through Mode........................67
3.4.Using GPU Pass-Through on Microsoft Windows Server.....................................................69
3.4.1.Assigning a GPU to a VM on Microsoft Windows Server with Hyper-V..........................70
3.4.2.Returning a GPU to the Host OS from a VM on Windows Server with Hyper-V............. 71
3.5.Using GPU Pass-Through on VMware vSphere.................................................................... 72
Chapter4.Installing the NVIDIA vGPU Software Graphics Driver.................................... 74
4.1.Installing the NVIDIA vGPU Software Graphics Driver on Windows..................................... 74
4.2.Installing the NVIDIA vGPU Software Graphics Driver on Linux...........................................76
4.3.Disabling the Wayland Display Server Protocol for Red Hat Enterprise Linux.................... 79
Chapter5.Licensing an NVIDIA vGPU...............................................................................81
5.1.Licensing an NVIDIA vGPU on Windows................................................................................81
5.2.Licensing an NVIDIA vGPU on Linux......................................................................................83
Chapter6.Modifying a VM's NVIDIA vGPU Configuration................................................. 85
6.1.Removing a VM’s NVIDIA vGPU Configuration...................................................................... 85
6.1.1.Removing a Citrix Virtual Apps and Desktops VM’s vGPU configuration.......................85
6.1.1.1.Removing a VM’s vGPU configuration by using XenCenter..................................... 85
6.1.1.2.Removing a VM’s vGPU configuration by using xe...................................................86
6.1.2.Removing a vSphere VM’s vGPU Configuration..............................................................86
6.2.Modifying GPU Allocation Policy............................................................................................ 87
6.2.1.Modifying GPU Allocation Policy on Citrix Hypervisor....................................................87
6.2.1.1.Modifying GPU Allocation Policy by Using xe...........................................................87
6.2.1.2.Modifying GPU Allocation Policy GPU by Using XenCenter..................................... 88
6.2.2.Modifying GPU Allocation Policy on VMware vSphere....................................................88
6.3.Migrating a VM Configured with vGPU.................................................................................. 91
6.3.1.Migrating a VM Configured with vGPU on Citrix Hypervisor.......................................... 92
6.3.2.Migrating a VM Configured with vGPU on VMware vSphere.......................................... 92
6.3.3.Suspending and Resuming a VM Configured with vGPU on VMware vSphere.............. 94
6.4.Modifying a MIG-Backed vGPU's Configuration.................................................................... 95
Chapter7.Monitoring GPU Performance..........................................................................98
7.1.NVIDIA System Management Interface nvidia-smi............................................................... 98
Virtual GPU Software DU-06920-001 _v12.0 through 12.1|v
7.2.Monitoring GPU Performance from a Hypervisor.................................................................99
7.2.1.Using nvidia-smi to Monitor GPU Performance from a Hypervisor...............................99
7.2.1.1.Getting a Summary of all Physical GPUs in the System......................................... 99
7.2.1.2.Getting a Summary of all vGPUs in the System.....................................................100
7.2.1.3.Getting vGPU Details............................................................................................... 101
7.2.1.4.Monitoring vGPU engine usage...............................................................................101
7.2.1.5.Monitoring vGPU engine usage by applications.....................................................102
7.2.1.6.Monitoring Encoder Sessions................................................................................. 103
7.2.1.7.Monitoring Frame Buffer Capture (FBC) Sessions................................................ 104
7.2.1.8.Listing Supported vGPU Types................................................................................109
7.2.1.9.Listing the vGPU Types that Can Currently Be Created........................................ 110
7.2.2.Using Citrix XenCenter to monitor GPU performance................................................. 110
7.3.Monitoring GPU Performance from a Guest VM.................................................................111
7.3.1.Using nvidia-smi to Monitor GPU Performance from a Guest VM...............................112
7.3.2.Using Windows Performance Counters to monitor GPU performance....................... 113
7.3.3.Using NVWMI to monitor GPU performance................................................................ 114
Chapter8.Changing Scheduling Behavior for Time-Sliced vGPUs................................ 117
8.1.Scheduling Policies for Time-Sliced vGPUs........................................................................117
8.2.Scheduler Time Slice for Time-Sliced vGPUs..................................................................... 118
8.3.RmPVMRL Registry Key........................................................................................................118
8.4.Getting the Current Time-Sliced vGPU Scheduling Behavior for All GPUs........................120
8.5.Changing the Time-Sliced vGPU Scheduling Behavior for All GPUs..................................121
8.6.Changing the Time-Sliced vGPU Scheduling Behavior for Select GPUs............................122
8.7.Restoring Default Time-Sliced vGPU Scheduler Settings.................................................. 123
Chapter9.Troubleshooting..............................................................................................125
9.1.Known issues........................................................................................................................ 125
9.2.Troubleshooting steps.......................................................................................................... 125
9.2.1.Verifying the NVIDIA Kernel Driver Is Loaded.............................................................. 125
9.2.2.Verifying that nvidia-smi works..................................................................................... 126
9.2.3.Examining NVIDIA kernel driver output........................................................................ 126
9.2.4.Examining NVIDIA Virtual GPU Manager Messages.....................................................126
9.2.4.1.Examining Citrix Hypervisor vGPU Manager Messages.........................................126
9.2.4.2.Examining Red Hat Enterprise Linux KVM vGPU Manager Messages.................. 127
9.2.4.3.Examining VMware vSphere vGPU Manager Messages........................................ 128
9.3.Capturing configuration data for filing a bug report...........................................................128
9.3.1.Capturing configuration data by running nvidia-bug-report.sh................................... 128
9.3.2.Capturing Configuration Data by Creating a Citrix Hypervisor Status Report............. 129
AppendixA.Virtual GPU Types Reference.......................................................................131
- 1
- 2
前往页