JyBoss - Manage JBoss from Jython
A jython module to automate JBoss/Wildfly server configuration either via the jython shell or Ansible. The jyboss modules use the jboss scripting cli library.
Prerequisites
The target machine on which to run this module must have Java 8 and Jython 2.7 installed.
Installation
The module can be installed using pip
with Jython 2.7.x.
pip install -U jyboss
Alterantively one can also download the packaged JyBoss module and install it with pip (ensure the pip in use is the jython pip and not the python version which is typically installed with the operating system).
Limitations
Requires at least ansible 2.0.2.0 if running the module with become_user
due to an ansible bug issue #14348).
Usage
Given this module uses the JBoss CLI client package, one must make sure the jboss-cli-client.jar
is in the CLASSPATH
or that the JBOSS_HOME
variable is set in the environment.
Example Environment:
$ export JYTHON_HOME=/opt/jython-2.7.1
$ export JBOSS_HOME=/opt/jboss-eap-7.0
$ export PATH=$JYTHON_HOME/bin:$PATH
$ jython
>>>>
Start the server you like to manage, import the JyBoss, connect and hack away.
#! /usr/bin/env jython
from jyboss import *
embedded.connect()
cd('/subsystem=datasources/data-source')
ls()
'''
[
"ExampleDS",
"ApimanDS"
]
'''
cd('ExampleDS')
# you can use pretty much any of the commands possible on the normal cli
cmd(':read-resource(attributes-only=true, include-defaults=false)')
'''
{
"response": {
"connectable": false,
"track_statements": "NOWARN",
"connection_url": "jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE",
"share_prepared_statements": false,
...
}
}
'''
# to assign command output to variables, change to non-interactive mode
embedded.context.interactive=False
# issue any kind of command and assign the result to a variable
r = ls('/subsystem=datasources/data-source=ExampleDS')
print(r['children'])
'''
[u'connection-properties']
'''
# also ensure to disconnect when done (or use automatic resource management described below)
if embedded.context.is_connected():
embedded.disconnect()
JyBoss also imports is also a Connection resource that can be used to handle the connection lifecycle using pythons with resource management (works like the file modules).
Another feature is the non-interactive mode of the session. In this mode printing to the terminal is suppressed and one can use the command and assign the result to variables - think ansible modules where the stdout must be a valid json response at the end of script execution.
#! /usr/bin/env jython
import json
from jyboss import *
# connection takes the same arguments as the static connect() command
with standalone as conn:
# you now have a valid session
# jyboss context is available and so is the actual JBoss CLI
print "the jyboss connection context: ", conn.context
print "the jboss jcli: ", conn.jcli
# if you want to assign the dict/list values of the commands to variables
conn.context.interactive = False
cd('/')
r = ls()
print json.dumps(r, indent=4)
# your connection has now been closed, further reading I recommend
# http://effbot.org/zone/python-with-statement.htm
Advanced Configuration
Syslog
As the jyboss module is primarily written to be used with ansible it cannot write output to the terminal console in noninteractive mode. To be able to still troubleshoot things, the module is configured to write to the user facility in syslog if it is present. Just add below line to the /etc/rsyslog.conf
file and tail the /var/log/user
file for useful information.
# Log any user messages
user.* /var/log/user
Why You Ask?
A few weeks ago I tried to automate the installation and setup of JBoss AS with Ansible. Butchering standalone.xml files on initial deployment was not an option as the server xml files change as soon as one manages the server. I tried my luck with the jboss-cli but after reflecting on what this would look like in practice I quickly figured that this is not the right way to interact with the boss either. Simply getting facts on a datasource into ansible is a crazy commandline from hell.
# ansible task example which retrieves the datasource configuration of ExampleDS and makes the details available as facts.
- shell: >
$JBOSS_HOME/bin/jboss-cli.sh -c --command="/subsystem=datasources/data-source=ExampleDS:read-resource(include-defaults=false, include-runtime=true, recursive=true)" | \
sed -r -e 's/=>/:/g'
-e 's/undefined/null/g'
-e 's/"(type|value-type)"\s:\s([^"][A-Za-z0-9]*[^",])/"\1":"\2"/g'
-e 's/"(.*)"\s?:\s?([^"][0-9.]*)L/"\1":\2/g'
-e 's/:(\s?expression\s?)/:/g'
-e '/'\''?\[/!b;:a;/\]'\''?/bb;$!{N;ba};:b;s/('\''?\[.*)[ \t\n]+(.*\]'\''?)/\1\2/;tb' |
jq 'def walk(f): . as $in | if type == "object" then reduce keys[] as $key ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f elif type == "array" then map( walk(f) ) | f else f end; walk( if type == "object" then with_entries( .key |= sub( "-+"; "_") ) else . end )'
register: example_ds_check
- set_fact:
example_ds: "{{ example_ds_check.stdout | from_json }}"
So where from here? After some fiddling around with jython
, the jboss-cli-client.jar
and ansible
I managed to produce a custom ansible jython module. Things got much more sophisticated much more quickly from the initial hack jython script that I decised to separate the ansible module from the jython wrapper.
Example use of jboss-cli in Jython (the hard way)
Basic use of the CLI looks something like this:
from org.jboss.as.cli.scriptsupport import CLI
cli = CLI.newInstance()
cli.connect()
cmd = cli.cmd("ls /")
if cmd.isSuccess():
print cmd.getResponse()
Controlling Output
Running this cli as an Ansible module, we need to make sure to disable all system output. Ansible module execution will break if the output to stdout is not valid JSON. The JBoss cli is unfortunately very inconsistent and random information is echoed by the underlying jline cli. There are some delegation methods in the code but some are protected and even if butchered, some things still leak to stdout and stderr.
Below a simplified configuration to shut the cli up but the jyboss module also contains a set of Streams to redirect to syslog if available.
from java.lang import System
from java.io import PrintStream, OutputStream
class NoopOutputStream(OutputStream):
def __init__(self):
pass
def write(self, b):
pass
def write(self, b, off, len):
pass
def close(self):
pass
def close(flush):
pass
System.setOut(PrintStream(NoopOutputStream()))
System.setErr(PrintStream(NoopOutputStream()))
Building a Release
This project uses the standard python setup mechanism and is configured to be deployable to pypi. Build and release distributable packages as followed.
VER=0.2.4
GPG_KEY=<your key>
# first cleanup
rm -rf dist build jyboss.egg-info
# build source distribution
python setup.py sdist
# build wheel binary
python setup.py bdist_wheel
# sign distro
gpg --detach-sign -a --default-key $GPG_KEY dist/jyboss-$VER.tar.gz
gpg --detach-sign -a --default-key $GPG_KEY dist/jyboss-$VER-py2.py3-none-any.whl
# upload to test
twine upload -r pypitest dist/jyboss-$VER.tar.gz \
dist/jyboss-$VER.tar.gz.asc \
dist/jyboss-$VER-py2.py3-none-any.whl \
dist/jyboss-$VER-py2.py3-none-any.whl.asc
# validate bundle
export JYTHON_HOME=/opt/jython-2.7.1
export PATH=$JYTHON_HOME/bin:$PATH
# install from test library
pip install --index-url https://test.pypi.org/simple jyboss
# now test it
export JBOSS_HOME=/opt/keycloak-3.4.3
# upload to pypi production
twine upload -r pypi dist/jyboss-$VER.tar.gz \
dist/jyboss-$VER.tar.gz.asc \
dist/jyboss-$VER-py2.py3-none-any.whl \
dist/jyboss-$VER-py2.py3-none-any.whl.asc
Run this script to validate the module can load and attaches to a JBoss server.
#!/usr/bin/env jython
from jyboss import *
embedded.connect()
ls()
embedded.disconnect()
Dev Notes
Connect to a running JBoss Server
set "CLASSPATH=%JAVA_HOME%\lib\jconsole.jar"
set "CLASSPATH=%CLASSPATH%;%JAVA_HOME%\lib\tools.jar"
set "CLASSPATH=%CLASSPATH%;%cd%\tests\bin\client\jboss-cli-client.jar"
jconsole service:jmx:remote+http://wildfly.vagrant.v8:9990
Configuring PyCharm
Mark project root as source folder
Unit Test configuration
- Select Path :
{workspaces}/jyboss-cli/tests
- Working Directory:
{workspaces}/jyboss-cli
- Add context root to PYTHONPATH: unchecked
- Add sources root to PYTHONPATH: checked
Running Keycloak Tests
To run the keycloak unittest:
- install jython
java -jar ~/Downloads/jython-installer-2.7.1.jar -s -d /usr/local/opt/jython -t standard -e demo
- setup up keycloak server
Either download and unzip a keycloak server distribution and the wildfly adapter overlay according to the keycloak installation guide or use the create-test-server.sh
script which installs on into ./tmp/server
.
- Set environment to use the jboss home
Either set the JBOSS_HOME
environment variable or update the jboss-test.properties
file accordingly