Torque Resource Manager Administrator Guide 6.1.3

Torque Resource Manager

Administrator Guide 6.1.3

February 2021

© 2018, 2021 Adaptive Computing Enterprises, Inc. All rights reserved.

This documentation and related software are provided under a license agreement containing

restrictions on use and disclosure and are protected by intellectual property laws. Except as

expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce,

translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any

part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this

access to or use of third-party content, products, or services, except as set forth in an applicable

agreement between you and Adaptive Computing.

Adaptive Computing, Moab®, Moab HPC Suite, Moab Viewpoint, Moab Grid, NODUS Cloud OS™, and

other Adaptive Computing products are either registered trademarks or trademarks of Adaptive

Computing Enterprises, Inc. The Adaptive Computing logo is a trademark of Adaptive Computing

Enterprises, Inc. All other company and product names may be trademarks of their respective

companies.

The information contained herein is subject to change without notice and is not warranted to be

error free. If you find any errors, please report them to us in writing.

Adaptive Computing Enterprises, Inc.

Welcome 14

Chapter 1: Introduction 15

1.1 Torque Administrator Guide Overview 15

1.2 Getting Started 17

1.2.1 What Is A Resource Manager? 17

1.2.2 What Are Batch Systems? 17

1.2.3 Basic Job Flow 18

Chapter 2: Installation And Configuration 20

2.1 Torque Installation Overview 22

2.2 Basic Server Configuration 23

2.4.1 Requirements 27

2.4.2 Open Necessary Ports 28

2.4.3 Install Dependencies, Packages, Or Clients 29

2.4.4 Install Torque Server 31

2.4.5 Install Torque MOMs 33

2.4.6 Install Torque Clients 34

2.4.7 Configure Data Management 35

2.5 Compute Nodes 37

2.6 Enabling Torque As A Service 39

2.7 Initializing/Configuring Torque On The Server (pbs_server) 40

2.12 Finalizing Configurations 53

2.13 Advanced Configuration 54

2.14 Customizing The Install 55

2.30.1 Configure Server_priv/nodes 99

2.30.2 Edit The /etc/hosts File 99

2.30.3 Start Pbs_mom With Multi-MOM Options 100

2.31 Stopping Pbs_mom In Multi-MOM Mode 101

3.3 Managing Multi-Node Jobs 108

3.4 Requesting Resources 109

3.4.1 Native Torque Resources 109

3.4.2 Interpreting Resource Requests 116

3.4.3 Interpreting Node Requests 116

3.4.4 Moab Job Extensions 117

3.5 Requesting NUMA-Aware Resources 119

3.6 Requesting Generic Resources 120

3.7 Requesting Floating Resources 121

3.8 Requesting Other Resources 122

3.15 Job Preemption 132

3.16 Keeping Completed Jobs 133

3.17 Job Checkpoint And Restart 134

3.18 Introduction To BLCR 135

3.19 Configuration Files And Scripts 136

3.20 Starting A Checkpointable Job 141

3.21 Checkpointing A Job 143

3.22 Restarting A Job 144

3.22.1 Restarting A Job In The Held State 144

3.22.2 Restarting A Job In The Completed State 144