zeppelin-0.8.0-bin-all.tgz

Advanced Topics =============== Accessing Java collections and arrays from Python ------------------------------------------------- Java collections are automatically mapped to Python collections so that standard Python operations such as slicing work on Java collections. Here is the mapping of the collection: =================== ====================== ========================================================== Java Collection Python Collection Py4J Implementation =================== ====================== ========================================================== Array Sequence [#arraynote]_ :class:`JavaArray <py4j.java_collections.JavaArray>` java.util.List MutableSequence :class:`JavaList <py4j.java_collections.JavaList>` java.util.Set MutableSet :class:`JavaSet <py4j.java_collections.JavaSet>` java.util.Map MutableMapping :class:`JavaMap <py4j.java_collections.JavaMap>` java.util.Iterator *Iterator Protocol* :class:`JavaIterator <py4j.java_collections.JavaIterator>` =================== ====================== ========================================================== .. [#arraynote] Py4J allows elements to be modified (like a real Java array), which is not the case of true immutable sequences like tuples. Java methods are still accessible when using the Python version of a Java collection. Here are some usage examples for each collection class. These examples do not cover the entire API. It is possible to change this default mapping by registering an output converter with :func:`register_output_converter <py4j.protocol.register_output_converter>` Array ^^^^^ :: >>> gateway = JavaGateway() >>> int_class = gateway.jvm.int >>> int_array = gateway.new_array(int_class,2) >>> int_array[0] = 1 >>> int_array[1] = 2 >>> int_array[0] 1 >>> int_array[2] Traceback (most recent call last): ... IndexError: list index out of range >>> for i in int_array: ... print(i) ... 1 2 >>> sarray = gateway.new_array(gateway.jvm.java.lang.String,2,3) >>> len(sarray) 2 >>> len(sarray[0]) 3 >>> sarray[0][1] = 'hello' >>> sarray[0][1] u'hello' >>> sarray[0][0] == None True List ^^^^ :: >>> l = gateway.jvm.java.util.ArrayList() >>> l.append(1) # calling Python interface >>> l.add('hello') # calling Java interface >>> for elem in l: ... print elem ... 1 hello >>> l[0] = 2 >>> l.append(3) >>> str(l) "[2, u'hello', 3]" >>> l2 = l[0:-1] >>> l2[0] = 999 >>> l [2, u'hello', 3] >>> l2 # l2 is a copy of l and not a view so a change in l2 does not affect l [999, u'hello'] >>> del(l[0]) >>> l [u'hello', 3] Set ^^^ :: >>> s = gateway.jvm.java.util.HashSet() >>> s.add(1) >>> s.add('hello') >>> s set([1, u'hello']) >>> 1 in s True >>> s.remove(u'hello') >>> s set([1]) Map ^^^ :: >>> m = gateway.jvm.java.util.HashMap() >>> m["a"] = 0 >>> m.put("b",1) >>> m {u'a': 0, u'b': 1} >>> u"b" in m True >>> del(m["a"]) >>> m {u'b': 1} >>> m["c"] = 2 >>> for key in m: ... print("%s:%i" % (key,m[key])) ... b:1 c:2 Byte array (byte[]) ------------------- Since version 0.7, Py4J automatically passes Java byte array (i.e., `byte[]`) by value and convert them to Python bytearray (2.x) or bytes (3.x) and vice versa. The rationale is that byte array are often used for binary processing and are often immutable: a program reads a series of byte from a data source and interpret it (or transform it into another byte array). Prior to 0.7, a Python program could interact with a Java byte[] but each access to a byte required a call between the Python and the Java interpreter (i.e., the Python program only had a reference to the array). In summary: * If from Java, you return a byte[], Py4J will convert the byte[] to a bytearray (Python 2.x) or bytes (Python 3.x) variable in Python. * If from Python, you pass a bytearray or bytes variable to the Java side, Py4J will convert it to a byte[]. * If you pass a byte[] from Java to Python, both arrays are disconnected: a change to the array on one side will not be reflected on the other side. * If you want to pass an array by reference, use a `Byte[]` instead of a `byte[]` or you can use `Bytes.asList <http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/primitives/Bytes.html#asList(byte...)>`_ from `Guava <http://code.google.com/p/guava-libraries/>`_ to obtain a list backed by a byte array. Implementing Java interfaces from Python (callback) --------------------------------------------------- Since version 0.3, Py4J allows Python classes to implement Java interfaces so that the JVM can call back Python objects. In the following example, a Python class implements a Java listener interface. Here is the code of listener interface: .. code-block:: java // py4j/examples/ExampleListener.java package py4j.examples; public interface ExampleListener { Object notify(Object source); } Here is the code of the main Java application. The program has a main method starting a `GatewayServer`. The entry point, a `ListenerApplication` instance, has two methods, one to register listeners, the other to notify all listeners. Start this program before running the Python code below. .. code-block:: java // py4j/examples/ListenerApplication.java package py4j.examples; import py4j.GatewayServer; import java.util.ArrayList; import java.util.List; public class ListenerApplication { List<ExampleListener> listeners = new ArrayList<ExampleListener>(); public void registerListener(ExampleListener listener) { listeners.add(listener); } public void notifyAllListeners() { for (ExampleListener listener: listeners) { Object returnValue = listener.notify(this); System.out.println(returnValue); } } @Override public String toString() { return "<ListenerApplication> instance"; } public static void main(String[] args) { ListenerApplication application = new ListenerApplication(); GatewayServer server = new GatewayServer(application); server.start(true); } } Here is the Python program that implements the Java interface and calls the ListenerApplication to notify all listeners. Then multiple exchanges between Java and Python occurs (e.g., Python calls System.out.println on the Java side). :: from py4j.java_gateway import JavaGateway, CallbackServerParameters class PythonListener(object): def __init__(self, gateway): self.gateway = gateway def notify(self, obj): print("Notified by Java") print(obj) gateway.jvm.System.out.println("Hello from python!") return "A Return Value" class Java: implements = ["py4j.examples.ExampleListener"] if __name__ == "__main__": gateway = JavaGateway( callback_server_parameters=CallbackServerParameters()) listener = PythonListener(gateway) gateway.entry_point.registerListener(listener) gateway.entry_point.notifyAllListeners() gateway.shutdown() The `PythonListener` class is a standard Python class that has one method, `notify`. The signature of the method contains one parameter. When interfaces contain overloaded methods, the python method must accept all possible combinations of parameters (with `*args` and `**kwargs` or with default parameters). Py4J recognizes that the `PythonListener` class implements a Java interface because it declares an internal class called `Java`, which has a member named `implements`. This member is a list of string representing the fully qualified name of implemented Java interfaces. Finally, the Python program contains a main method that starts a gateway, initializes a PythonListener instance and registers it to the `Listene