Fast remote procedure calls to Python; introducing a new JSON-RPC stream protocol

Estimated reading time of this article: 6 minutes 58 seconds

How to call a Python function from Java?

I develop a solar energy simulation software (Polysun). Controllers can be written as plugins. The SimpleRpcPluginController allows to implement controller logic in other programming languages.

Functions from controlFunctions.py starting with "control_" should be callable from Java.

As shown in article Remote procedure calls to Python using XML-RPC and JSON-RPC with performance comparision standard XML-RPC and standard JSON-RPC show a poor performance for our simulation tool, where Python functions are called more than 100'000 times from Java. The problem is the HTTP protocol where for each request a new TCP connection is created. On Windows, JSON-RPC cannot even be used as the client runs into "java.net.BindException: Address already in use: connect" due to ephemeral TCP ports exhaustion.

I created a new TCP protocol basing on JSON-RPC 2.0, the JSON-RPC stream protocol.

JSON-RPC stream protocol

One TCP connection per simulation is used.

The simple custom RPC streaming protocol sends JSON-RPC 2.0 string lines encoded as UTF-8:

  1. Open TCP socket communication connection
  2. Send JSON-RPC 2.0 request terminated by EOL.
  3. Read JSON-RPC 2.0 response terminated by EOL.
  4. Repeat to step 2 until simulation has ended.
  5. (Optional) Send empty line terminated by EOL to terminate the connection.

EOL = End of line, i.e. \n in Python/Java or Unicode \u000a

It is not anymore JSON-RPC 2.0 compatible as the communication is not over HTTP.

Measurements

Measured average time per function call for contro_flowrate() in Python and FlowratePluginController in Java 8 (Eclipse debug) on a Intel Core i7-4500U CPU 1.80GHz (dual core) for more than 100'000 function calls on localhost are about:

JSON-RPC stream JSON-RPC XML-RPC Java Matlab
RPC type: custom standard standard native library
Communication protocol: TCP socket HTTP HTTP native RMI
TCP connection per simulation timestep timestep N/A simulation
PyPy 5.4.1, Java non-debug 0.06ms 0.3ms 0.6ms 0ms 0.6ms
PyPy 5.4.1 0.06ms 0.5ms 0.9ms 0ms 0.6ms
Python 2.7 0.09ms 0.7ms 1.3ms 0ms 0.6ms
Python 3.4 0.08ms 0.9ms 1.4ms 0ms 0.6ms
Jython 2.7.0 (20ms)

Performance factors comparing for the same Python interpreter (e.g JSON-RPC using pypy needs five times (500%) more time than JSON-RPC stream):

JSON-RPC stream JSON-RPC XML-RPC Java Matlab
PyPy 5.4.1, Java non-debug 1 5 10 0 10
PyPy 5.4.1 1 8 15 0 10
Python 2.7 1 8 14 0 7
Python 3.4 1 11 18 0 8

Performance factors comparing with JSON-RPC stream using pypy (e.g XML-RPC on python 3.3 needs 23 times (2300%) more time):

JSON-RPC stream JSON-RPC XML-RPC Java Matlab
PyPy 5.4.1, Java non-debug 1.0 5 10 0 10
PyPy 5.4.1 1.0 8 15 0 10
Python 2.7 1.5 12 22 0 10
Python 3.4 1.3 15 23 0 10

The JSON-RPC stream protocol on pypy is five times faster (500%!) than the standard JSON-RPC HTTP-based RPC protocol on pypy. The JSON-RPC stream on pypy is 23 times (2300%!) faster than the XML-RPC on python 3.4.

This function is called more than 100'000 times from Java code.

On the Java side a TCP socket and parts of JSON-RPC for JSON-RPC function wrapping are used.

The scripts work with Python 2.7 and Python 3.4.

Call: python controlJsonRpcStreamServer.py or: pypy controlJsonRpcStreamServer.py

Supports pypy (http://pypy.org/) with JIT compliation

JSON-RPC stream server

#!/usr/bin/env python
"""
JSON-RPC Server for Polysun plugin controller functions of SimpleRpcPluginController (RPC-Type: JSON-RPC stream).
"""

from __future__ import division, unicode_literals, print_function, absolute_import, with_statement  # Ensure compatibility with Python 3
import argparse
import inspect
import sys
import json
import threading
import request_jsonrpc
import importlib
from utils import indent

PY3 = sys.version_info[0] == 3
PY2 = sys.version_info[0] == 2

if PY3:
    from socketserver import StreamRequestHandler
    from socketserver import TCPServer
elif PY2:
    from SocketServer import StreamRequestHandler
    from SocketServer import TCPServer

__author__ = 'Roland Kurmann'
__email__ = 'roland dot kurmann at velasolaris dot com'
__url__ = 'http://velasolaris.com'
__license__ = 'MIT'
__version__ = '9.2'

parser = argparse.ArgumentParser(description='Polysun control function (JSON-RPC stream server using TCP).')
parser.add_argument('-p', '--port', default='2102', type=int, help='Port of the JSON-RPC stream server (TCP socket). Default 2102')
parser.add_argument('-s', '--host', default='127.0.0.1', help='Host address of the stream server. Default 127.0.0.1')
parser.add_argument('-f', '--functions', default='controlFunctions', help='Python module with control functions. Default controlFunctions')
parser.add_argument('-d', '--debug', action='store_true', help='Enable debug mode with debug output')

args = parser.parse_args()

# instead of 'import controlFunctions', load dynamically using arguments
importlib.import_module(args.functions)

class JsonRpcStreamServerHandler(StreamRequestHandler):
    """
    The request handler class for our TCP server.

    It is instantiated once per connection to the server, and must
    override the handle() method to implement communication to the
    client.

    See https://docs.python.org/2/library/socketserver.html
    """
    def handle(self):
        try:
            print("Connection opened from {} and listening...".format(self.client_address[0]))
            ok = True
            while ok:
                # self.rfile is a file-like object created by the handler;
                # we can now use e.g. readline() instead of raw recv() calls
                self.data = self.rfile.readline().strip()
                # print "{} wrote:".format(self.client_address[0])
                if args.debug:
                    print("'" + self.data + "'")
                # Likewise, self.wfile is a file-like object used to write back
                # to the client
                # self.wfile.write(self.data.upper())
                jsonResponse = self.data.decode('UTF-8')
                # An empty line means stopping the connection
                if jsonResponse != None and jsonResponse.strip() != "":
                    request = json.loads(jsonResponse)
                    response = jsonrpc.handle_rpc(request)
                    jsonResponse = json.dumps(response) + "\n"
                    self.wfile.write(jsonResponse.encode('UTF-8'))
                    self.wfile.flush()
                    if args.debug:
                        print("handled: " + jsonResponse)
                else:
                    ok = False
            print("Connection  stopped")
        except KeyboardInterrupt:
            print("\nKeyboard interrupt received in request, exiting.")
            server_shutdown()

jsonrpc = request_jsonrpc.register(args.debug)

# echo '{ "jsonrpc": "2.0", "method": "ping", "params": [], "id": 1}' | nc 127.0.0.1 2102
@jsonrpc
def ping():
    """Returns "pong"."""
    return "pong"

# echo '{ "jsonrpc": "2.0", "method": "print", "params": ["Wow"], "id": 1}' | nc 127.0.0.1 2102
def print_msg(msg):
    """Prints a message on the server."""
    print(msg)

jsonrpc.methods['print'] = print_msg

def server_shutdown():
    """Shutdown TCP server.

    server.shutdown() must not be called in the main thread and sys.exit(0) does not work,
    and  os._exit(0) does not clean up server which leads to 'error: [Errno 98] Address already in use'
    """
    print("Initiate server shutdown...")
    threading.Thread(target=server.shutdown).start()

jsonrpc.methods['stop'] = server_shutdown

# http://stackoverflow.com/questions/4040620/is-it-possible-to-list-all-functions-in-a-module
functions = inspect.getmembers(sys.modules[args.functions], inspect.isfunction)
for function in functions:
    if function[0].startswith("control_"):
        jsonrpc.methods[function[0]] = function[1]

print("\nStart JSON-RPC stream server (TCP)...")
print("jsonrpc2://" + args.host + ":" + str(args.port))
li = "\n    - "
print("Functions:" + li + li.join(str(x) for x in sorted(jsonrpc.methods)))
print("Ctrl-C to stop")

server = None
try:
    # Create the server
    server = TCPServer((args.host, args.port), JsonRpcStreamServerHandler)
    server.allow_reuse_address = True

    # Activate the server; this will keep running until you interrupt the program with Ctrl-C
    server.serve_forever()
except KeyboardInterrupt:
    print("\nKeyboard interrupt received, exiting.")
    server.shutdown()
finally:
    print("Terminate server")
    if server != None:  # Check if server exists, since in case of server creation errors, server variable may not be set
        server.server_close()
    sys.exit(0)

    sys.exit(0)
"""
Very minimal implementation of JSON-RPC requests.

Adapted from bottle_jsonrpc.py (https://github.com/olemb/bottle_jsonrpc).
Based on bottle_jsonrpc version 0.2.1 of Ole Martin Bjorndalen.

"""
from __future__ import division, unicode_literals, print_function, absolute_import, with_statement  # Ensure compatibility with Python 3
import sys, traceback

def get_public_methods(obj):
    """Return a dictionary of all public callables in a namespace.

    This can be used for objects, classes and modules.
    """
    methods = {}

    for name in dir(obj):
        method = getattr(obj, name)
        if not name.startswith('_') and callable(method):
            methods[name] = method

    return methods


class NameSpace:
    def __init__(self, debug=False, obj=None, catchall=True):
        self.debug = debug
        self.methods = {}
        self.catchall = catchall

        if obj is not None:
            self.add_object(obj)

    def add_object(self, obj):
        """Adds all public methods of the object."""
        self.methods.update(get_public_methods(obj))

    def handle_rpc(self, request):
        try:
            name = request['method']
            func = self.methods[name]
            params = request.get('params', {})
            if params != None:
                result = func(*params)
            else:
                result = func()

            return {
                'jsonrpc': '2.0',  # Added by rkurmann for JSON-RPC 2.0 compliancy
                'id': request['id'],
                'result': result,
                # 'error': None,  # Removed by rkurmann for JSON-RPC 2.0 compliancy
            }
        except:
            if not self.catchall:
                raise
            traceback.print_exc(file=sys.stderr)
            response = {
                'id': request['id'],
                # 'result': None,  # Removed by rkurmann for JSON-RPC 2.0 compliancy
                'error': 'Internal server error',
            }
            if self.debug:
                response['traceback'] = traceback.format_exc()

            return response

    def __call__(self, func):
        """This is called when the mapper is used as a decorator."""
        self.methods[func.__name__] = func
        return func

register = NameSpace
"""
Utils package
"""

from __future__ import division, unicode_literals, print_function, absolute_import, with_statement  # Ensure compatibility with Python 3

# Example:
# @static_vars(counter=0)
# def foo():
#     foo.counter += 1
#     print "Counter is %d" % foo.counter
# http://stackoverflow.com/questions/279561/what-is-the-python-equivalent-of-static-variables-inside-a-function
def static_vars(**kwargs):
    '''Annotation for static variables in functions. Similar to static in Java or persistent in Matlab.

    Works only for a single thread.
    '''
    def decorate(func):
        for k in kwargs:
            setattr(func, k, kwargs[k])
        return func
    return decorate

def indent(lines, padding="\t"):
    "Indent each line with padding"
    padding = padding
    return padding + ('\n' + padding).join(lines.split('\n'))

Conclusion

The JSON-RPC stream protocol is much better suited than the standard JSON-RPC using HTTP for many repeated function calls on a localhost.

Files: