Python to Clojure Bridge

Python to Clojure Bridge using a Py4J Gateway. This simple library aims to make it easy and trivial to invoke Clojure from Python using the py4j gateway server and API.

If you think this sounds convoluted, it is. However it also solves a lot of issues using a JVM API and proxying in the same process. This method separates the JVM and python in separate processes so they don't step on eachother's feet (think memory and resource allocation issues). On the downside the setup is more complex.

The end to end request looks like:

1. Invoke the clojure Python library.
2. Marshall the RPC request via the py4j python library
3. Request is sent via the network
4. Request received py4j Java Gateway server
5. zensols.py4j.gateway (this library) invokes the actual Clojure request
6. Return the result back all the way up the chain.

Table of Contents

• Usage
  • NLP Complex Example
  • Installing and Running
• Obtaining
• Documentation
• Binaries
• Building
• Changelog
• License

Usage

First download, install and run the server

from zensols.clojure import Clojure

def test():
    cw = Clojure('taoensso.nippy')
    try:
        cw.add_depenedency('com.taoensso', 'nippy', '2.13.0')
        dat = cw.invoke('freeze', [123, 'strarg', 1.2])
        thawed = cw.invoke('thaw', dat)
        for i in thawed:
            print('thawed item: %s' % i)
    finally:
        cw.close()

>>> test()
>>> thawed item: 123
thawed item: strarg
thawed item: 1.2

See the test cases for more examples.

NLP Complex Example

This example uses the NLP Clojure Project parse function. The py4jgw (gateway) needs the models installed and the following system property set by adding it to the environment setup script:

$ echo 'JAVA_OPTS="-Dzensols.model=${HOME}/opt/nlp/model"' > py4jgw/bin/setupenv
$ /bin/bash py4jgw/bin/py4jgw

The following example parses an utterance and prints out what could be used as features in a machine learning model:

import json
from zensols.clojure import Clojure

def test():
    parse = Clojure('zensols.nlparse.parse')
    cjson = Clojure('clojure.data.json')
    try:
        parse.add_depenedency('com.zensols.nlp', 'parse', '0.1.4')
        cjson.add_depenedency('org.clojure', 'data.json', '0.2.6')
        panon = parse.invoke('parse', """I LOVE Bill Joy and he's the smartest guy in the world!""")
        jstr = cjson.invoke('write-str', panon)
        parsed = json.loads(jstr)
        print('sentiment: %s' % parsed['sentiment'])
        ment = parsed['mentions'][0]
        print("'%s' is a %s" % (ment['text'], ment['ner-tag']))
        #print(json.dumps(parsed, indent=2, sort_keys=True))
    finally:
        parse.close()
        cjson.close()

>>> test()
>>> sentiment: 1
'Bill Joy' is a PERSON
{
  "mentions": [
    {
      "char-range": [
        7, 
        15
      ], 
	  ...

Installing and Running

1. Download the binary: $ wget https://github.com/plandes/clj-py4j/releases/download/v0.0.1/py4jgw.zip
2. Extract: $ unzip py4jgw.zip
3. Run the server: $ /bin/bash ./py4jgw/bin/py4jgw (or py4jgw\bin\py4jgw.bat)
4. Install the Python library: $ pip install zensols.clojure
5. Hack!

Obtaining

In your project.clj file, add:

Binaries

The latest release binaries are available here.

Documentation

• Clojure
• Java

Building

To build from source, do the folling:

• Install Leiningen (this is just a script)
• Install GNU make
• Install Git
• Download the source: git clone https://github.com/plandes/clj-py4j && cd clj-py4j
• Download the make include files:

mkdir ../clj-zenbuild && wget -O - https://api.github.com/repos/plandes/clj-zenbuild/tarball | tar zxfv - -C ../clj-zenbuild --strip-components 1

• Build the software: make jar
• Build the distribution binaries: make dist
• Build the Python egg/wheel distribution libraries: make pydist

Note that you can also build a single jar file with all the dependencies with: make uber

Changelog

An extensive changelog is available here.

License

Copyright © 2017 Paul Landes

Apache License version 2.0

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.