JPPF Manual
Table of Contents
1 Introduction.......................................................................5
1.1 Intended audience.......................................................5
1.2 Prerequisites................................................................5
1.3 Where to download......................................................5
1.4 Installation....................................................................5
1.5 Running the standalone modules.................................5
2 Tutorial : A first taste of JPPF............................................6
2.1 Required software........................................................6
2.2 Overview......................................................................6
2.2.1 Tutorial organization................................................6
2.2.2 Expectations...........................................................6
2.3 Writing a JPPF task.....................................................7
2.4 Creating and executing a job.......................................8
2.4.1 Creating and populating a job.................................8
2.4.2 Executing a job and processing the results.............8
2.5 Running the application..............................................11
2.6 Dynamic deployment.................................................12
2.7 Job Management.......................................................12
2.7.1 Preparing the job for management........................12
2.8 Conclusion.................................................................15
3 JPPF Overview...............................................................16
3.1 Architecture and topology..........................................16
3.2 Work distribution........................................................17
3.3 Networking considerations.........................................18
3.3.1 Two channels per connection................................18
3.3.2 Synchronous networking.......................................18
3.3.3 Protocol.................................................................19
3.4 Sources of parallelism................................................20
3.4.1 At the client level...................................................20
3.4.2 At the server level.................................................20
3.4.3 At the node level...................................................20
4 Development Guide........................................................21
4.1 Task objects...............................................................21
4.1.1 JPPFTask..............................................................21
4.1.2 Exception handling - node processing..................24
4.1.3 Executing code in the client from a task................24
4.1.4 JPPF-annotated tasks...........................................25
4.1.5 Runnable tasks.....................................................26
4.1.6 Callable tasks........................................................26
4.1.7 POJO tasks...........................................................26
4.1.8 Running non-Java tasks: CommandLineTask.......27
4.1.9 The Location API...................................................29
4.2 Dealing with jobs........................................................30
4.2.1 Job name and identifier.........................................30
4.2.2 Creating a job........................................................31
4.2.3 Adding tasks to a job.............................................31
4.2.4 Non-blocking jobs.................................................32
4.3 Jobs runtime behavior, recovery and failover.............34
4.3.1 Handling of execution results................................34
4.3.2 Failover and job re-submission.............................34
4.3.3 Job persistence and recovery...............................35
4.3.4 A full TaskResultListener example.........................36
4.3.5 Job start and completion notifications...................37
4.4 Sharing data among tasks : the DataProvider API.....38
4.4.1 MemoryMapDataProvider: map-based provider.. .38
4.4.2 ClientDataProvider: computing data in the client
(deprecated)..................................................................39
4.4.3 Data provider for non-JPPF tasks.........................40
4.5 Job Service Level Agreement....................................41
4.5.1 Attributes common to server and client side SLAs 41
4.5.2 Server side SLA attributes....................................45
4.5.3 Client side SLA attributes......................................47
4.6 Job Metadata.............................................................47
4.7 The JPPFClient API...................................................48
4.7.1 Creating and closing a JPPFClient.......................48
4.7.2 Submitting a job....................................................49
4.7.3 Cancelling a job....................................................49
4.7.4 Exploring the server connections..........................49
4.7.5 Receiving notifications for new connections.........49
4.7.6 Status notifications for existing connections.........50
4.7.7 Switching local execution on or off........................50
4.8 JPPF Executor Services............................................51
4.8.1 Basic usage..........................................................51
4.8.2 Batch modes.........................................................52
4.8.3 Configuring jobs and tasks....................................53
4.8.4 JPPFCompletionService.......................................54
4.9 The JPPF configuration API.......................................55
5 Configuration guide.........................................................56
5.1 Configuration file specification and lookup.................56
5.2 Reminder: JPPF topology..........................................57
5.3 Configuring a JPPF server.........................................58
5.3.1 Basic network configuration..................................58
5.3.2 Server discovery...................................................58
5.3.3 Connecting to other servers..................................58
5.3.4 Socket connections idle timeout............................59
5.3.5 JMX management configuration...........................59
5.3.6 Load-balancing.....................................................60
5.3.7 Server process configuration................................62
5.3.8 Configuring a local node.......................................62
5.3.9 Recovery from hardware failures of nodes...........62
5.3.10 Parallel I/O..........................................................63
5.3.11 Full server configuration file (default values).......63
5.4 Node configuration.....................................................65
5.4.1 Server discovery...................................................65
5.4.2 Manual network configuration...............................65
5.4.3 Socket connections idle timeout............................66
5.4.4 JMX management configuration...........................66
5.4.5 Recovery and failover...........................................66
5.4.6 Interaction of failover and server discovery...........66
5.4.7 Recovery from hardware failures..........................67
5.4.8 Processing threads...............................................67
5.4.9 Node process configuration..................................67
5.4.10 Class loader cache.............................................67
5.4.11 Class loader resources cache.............................69
5.4.12 Security policy.....................................................69
5.4.13 Full node configuration file (default values).........70
5.5 Client and administration console configuration.........71
5.5.1 Server discovery...................................................71
5.5.2 Manual network configuration...............................71
5.5.3 Using manual configuration and server discovery
together..........................................................................72
5.5.4 Recovery and failover...........................................72
5.5.5 Socket connections idle timeout............................73
5.5.6 Local and remote execution..................................73
5.5.7 Local execution flow..............................................73
5.5.8 Load-balancing in the client..................................74
5.5.9 Full client configuration file (default values)..........74
5.6 Configuring SSL/TLS communications......................76
5.6.1 Enabling secure connectivity................................76
5.6.2 Locating the SSL configuration.............................77
5.6.3 SSL configuration properties.................................78
5.6.4 Full SSL configuration example............................79
6 Management and monitoring..........................................80
6.1 Node management....................................................80
6.1.1 Node-level management and monitoring MBean. .80
6.1.2 Task-level monitoring............................................82
6.1.3 Accessing and using the node MBeans................83
6.1.4 Remote logging.....................................................85
6.2 Server management..................................................87
6.2.1 Server-level management and monitoring............87
6.2.2 Job-level management and monitoring.................90
6.2.3 Accessing and using the server MBeans..............92
6.2.4 Remote logging.....................................................94
7 Extending and Customizing JPPF...................................96
7.1 Pluggable MBeans.....................................................96
7.1.1 Elements and constraints common to node and
server MBeans...............................................................96
7.1.2 Writing a custom node MBean..............................96
7.1.3 Writing a custom server MBean............................98
7.2 JPPF startup classes...............................................100
7.2.1 Node startup classes..........................................100
7.2.2 Server startup classes........................................101
7.3 Transforming and encrypting networked data..........102
7.4 Specifying alternate object streams.........................105
7.4.1 Specifying the object stream implementation
classes.........................................................................105
7.4.2 Implementing an object stream builder...............105
7.4.3 Built-in implementations......................................105
7.5 Creating a custom load-balancer.............................107
7.5.1 Overview of JPPF load-balancing.......................107
7.5.2 Implementing the algorithm and its profile..........108
7.5.3 Implementing the bundler provider interface.......110
7.5.4 Deploying the custom load-balancer...................110
7.5.5 Node-aware load balancers................................110
7.5.6 Job-aware load balancers...................................111
7.6 Receiving node connection events in the server......112
7.7 Receiving notifications of node life cycle events......113
7.7.1 NodeLifeCycleListener interface.........................113
7.7.2 Extended notifications: the
NodeLifeCycleListenerEx interface..............................114
7.8 Node initialization hooks...........................................115
7.9 Fork/Join thread pool in the nodes...........................116
7.10 Flow of customizations in JPPF.............................118
7.10.1 JPPF driver.......................................................118
7.10.2 JPPF node........................................................118
8 Class Loading In JPPF..................................................119
8.1 How it works.............................................................119
8.2 Class loader hierarchy in JPPF nodes.....................120
8.3 Relationship between UUIDs and class loaders......121
8.4 Built-in optimizations................................................122
8.4.1 Deployment to specific grid components............122
8.4.2 Using a constant JPPF client UUID....................122
8.4.3 Node class loader cache.....................................122
8.4.4 Local caching of network resources....................122
8.4.5 Classes cache in the JPPF server......................123
8.4.6 Node customizations...........................................123
8.5 Class loader delegation models...............................123
8.6 JPPF class loading extensions................................125
8.6.1 Dynamically adding to the classpath...................125
8.6.2 Downloading multiple resources at once............125
8.6.3 Resources lookup on the file system..................125
8.7 Related sample........................................................125
9 J2EE Connector............................................................126
9.1 Overview of the JPPF Resource Adapter.................126
9.1.1 What is it?...........................................................126
9.1.2 Features..............................................................126
9.1.3 Architecture.........................................................126
9.2 Supported Platforms................................................127
9.3 Configuration and build............................................127
9.3.1 Requirement.......................................................127
9.3.2 Build structure.....................................................127
9.3.3 Building the JPPF resource adapter...................127
9.3.4 Configuring the resource adapter and demo
application....................................................................128
9.4 How to use the connector API..................................129
9.4.1 Obtaining and closing a resource adapter
connection...................................................................129
9.4.2 Submitting jobs...................................................130
9.4.3 Getting the status and results of a job.................130
9.4.4 Cancelling a job..................................................131
9.4.5 Synchronous execution.......................................131
9.4.6 Using submission status events..........................131
9.5 Deployment on a J2EE application server...............133
9.5.1 Deployment on JBoss 4.x - 6.x...........................133
9.5.2 Deployment on JBoss 7+....................................133
9.5.3 Deployment on Oracle OC4J 10.3.1.x................136
9.5.4 Deployment on SunAS / Glassfish......................140
9.5.5 Deployment on Websphere.................................147
9.5.6 Deployment on Weblogic....................................153
9.5.7 Deployment on Apache Geronimo......................161
9.6 Packaging your enterprise application.....................162
9.7 Creating an application server port..........................162
10 GigaSpaces XAP Connector.......................................164
10.1 About GigaSPaces and JPPF................................164
10.2 Installation..............................................................164
10.2.1 Prerequisites.....................................................164
10.2.2 Installing............................................................164
10.3 Deploying the JPPF processing unit......................164
10.4 Deploying the sample web application...................166
10.5 Considerations for deploying JPPF-enabled
applications....................................................................167
10.5.1 Available APIs...................................................167
10.5.2 Spring descriptor...............................................167
10.5.3 Usage in application code.................................168
11 Configuration properties reference..............................170
11.1 Server properties....................................................170
11.2 Node properties......................................................171
11.3 Application client and admin console properties.....172
12 Execution policy reference..........................................173
12.1 Execution Policy Elements.....................................173
12.1.1 NOT..................................................................173
12.1.2 AND..................................................................173
12.1.3 OR....................................................................173
12.1.4 XOR..................................................................174
12.1.5 Equal.................................................................174
12.1.6 LessThan..........................................................174
12.1.7 AtMost...............................................................175
12.1.8 MoreThan.........................................................175
12.1.9 AtLeast..............................................................175
12.1.10 BetweenII........................................................176
12.1.11 BetweenIE.......................................................176
12.1.12 BetweenEI.......................................................176
12.1.13 BetweenEE.....................................................177
12.1.14 Contains..........................................................177
12.1.15 OneOf.............................................................177
12.1.16 RegExp...........................................................178
12.1.17 CustomRule....................................................178
12.2 Execution policy properties....................................179
12.2.1 Related APIs.....................................................179
12.2.2 JPPF configuration properties...........................179
12.2.3 System properties.............................................179
12.2.4 Environment variables......................................179
12.2.5 Runtime properties............................................180
12.2.6 Network properties............................................180
12.2.7 Storage properties............................................180
12.3 Execution policy XML schema...............................182
13 Deployment and run modes........................................185
13.1 JPPF Driver............................................................185
13.1.1 JPPF Driver as a Windows Service..................185
13.1.2 JPPF Driver as a Linux/Unix daemon...............185
13.2 JPPF Node.............................................................185
13.2.1 JPPF Node as a Windows Service...................185
13.2.2 JPPF Node as a Linux/Unix daemon................186
13.2.3 JPPF Node in “Idle Host” mode........................186
13.3 Running JPPF on Amazon's EC2..........................187
13.3.1 Server discovery...............................................187
13.3.2 Firewall configuration........................................187
13.3.3 Instance type.....................................................187
13.3.4 IP Addresses.....................................................187
13.4 Runtime dependencies..........................................188
13.4.1 Node and Common dependencies....................188
13.4.2 Driver dependencies.........................................188
13.4.3 Client dependencies.........................................188
13.4.4 Administration console dependencies...............188
1 Introduction
1.1 Intended audience
This manual is intended for developers, software engineers and architects who wish to discover, learn or deepen their
knowledge of JPPF and how it works. The intent is also to provide enough knowledge to not only write your own
applications using JPPF, but also extend it by creating add-ons and connectors to other frameworks.
1.2 Prerequisites
JPPF works on any system that supports Java. There is no operating system requirement, it can be installed on all flavors
of Unix, Linux, Windows, Mac OS, and other systems such as zOS or other mainframe systems.
JPPF requires the following installed on your machine:
• Java Standard Edition version 1.5 or later, with the environment variable JAVA_HOME pointing to your Java
installation root folder
• Apache Ant, version 1.7.0 or later, with the environment variable ANT_HOME pointing to the Ant installation root
folder
• Entries in the default system PATH for JAVA-HOME/bin and ANT_HOME/bin
1.3 Where to download
All JPPF software can be downloaded from the JPPF downloads page.
We have tried to give each module a name that makes sense. The format is JPPF-x.y.z-<module-name>.zip, where:
• x is the major version number
• y is the minor version number
• z is the patch release number - it will not appear if no patch has been released (i.e. if it is equal to 0)
• <module-name> is the name given to the module
1.4 Installation
Each JPPF download is in zip format. To install it, simply unzip it in a directory of your choice.
When unzipped, the content will be under a directory called JPPF-x.y.z-<module-name>
1.5 Running the standalone modules
The JPPF distribution includes a number of standalone modules or components, which can be deployed and run
independantly from any other on separate machines, and/or from a separate location on each machine
These modules are the following:
• application template: this is the application template to use as starting point for a new JPPF application, file JPPF-
x.y.z-application-template.zip
• driver: this is the server component, file JPPF-x.y.z-driver.zip
• node: this is the node component, file JPPF-x.y.z-node.zip
• administration console: this is the management and monitoring user interface, file JPPF-x.y.z .admin-ui.zip
• multiplexer: this is the TCP multipler that routes all traffic through a single port, file JPPF-x.y.z .multiplexer.zip.
These can be run from either a shell script (except for the multiplexer) or an Ant script. The ant script is always called
"build.xml" and it always has a default target called "run". To run any of these modules, simply type "ant" or "ant run" in a
command prompt or shell console. The provided shell scripts are named start<Component>.<ext> where Component is
the JPPF component to run (e.g. “Node”, “Driver”, “Console”) and ext is the file extension, “bat” for Windows systems, or
“sh” for Linux/Unix-like systems.
