Let pop run your services!
Skip to acknowledgements and credits.
Software builds, deployment and service management, are tasks that become exceedingly difficult to carry out without the assistance of a computer.
Typically, this assistance is a collection of scripts that require some amount of handholding. When multiple machines are involved, this handholding becomes a problem of distributed coordination.
We need a tool!
To add to the confusion, it's also not meant to replace Supervisor, an excellent process control system written in Python. In fact, we recommend that you run your machine agents using this software.
What pop is meant to replace is you!
Now, we've already got a language – Python. All we need is to model the actions that you would carry out if you were the best possible system administrator.
With these principles we try to cover the basic design and philosophy.
- Adaptive. The system is dynamic and the equilibrium is the target state. There's always a state and if it's not the target state, then we must take the necessary steps to reach that state.
- Distributed. The system is comprised of services running on multiple machines. We use Apache's ZooKeeper for "highly reliable distributed coordination" (via txzookeeper, a Twisted library written for the Ubuntu Juju cloud deployment tool).
- Integrated. This is a Python-based tool for running Python-based services. The system is extensible through plugins.
- Open. The system should not be tied to a particular platform.
These steps will be necessary for each machine in the system – except for the final step which applies only to the machine that should run ZooKeeper.
We assume that Python 2.7 is installed and available as
$ wget http://peak.telecommunity.com/dist/ez_setup.py $ sudo python2 ez_setup.py $ sudo easy_install-2.7 virtualenv
That's it for
sudo. We can continue with an unprivileged user.
Let's install pop into an isolated, virtual environment under the home directory and enter the environment:
$ virtualenv --python=python2 ~/pop $ source ~/pop/bin/activate $ easy_install pop
This installs the
popcommand-line utility as well as supporting libraries.
Build and install ZooKeeper on the system (see instructions for more details).
$ cd ~/pop $ git clone git://git.apache.org/zookeeper.git $ cd zookeeper $ ant
First we need to build the C-bindings:
$ cd src/c $ autoreconf -if $ ./configure --prefix=~/pop/zookeeper $ make && make install
Then the Python-bindings:
$ cd ../contrib/zkpython $ ant install
That's it for the installation.
Configure and start ZooKeeper:
$ cd ~/pop/zookeeper/conf $ cp zoo_sample.cfg zoo.cfg
You should edit
zoo.cfgbefore you start a real deployment because the default storage setting is
There's a script included to run the service:
$ ../bin/zkServer.sh start
If you want to keep track of what's going on, use
start-foreground. Note that the
poputility expects ZooKeeper to run on
--hostargument is provided.
If everything's gone to plan, we're ready to use the system.
As a first look, in this tutorial we'll learn how to use pop to run a vanilla installation of Plone.
Initialize pop namespace:
$ pop init
This adds a number of nodes to the hierarchy that pop requires for its operation. Note that the command effectively resets the configuration although this requires the
--forceoption if an existing configuration is in place.
plone4service which runs the latest version of Plone:
$ pop add plone4
The service will be set up with the default configuration with the name
--nameargument was provided). We can confirm this using the
$ pop status plone4
In the default configuration, Plone is set to listen on port 8080. To change this, use the
--portparameter when you first add the service.
To make the local machine available as a system that we can run services on, we need to start the machine agent:
$ pop start
This process can also run in the foreground using
Finally, to deploy the Plone service on the local machine:
$ pop deploy plone4
This works because the utility assumes that we want to deploy the service on the local machine.
The order of the last two steps is not important. We could easily have deployed the service first, then made the machine available.
Pop keeps all state in ZooKeeper (ZK).
The table below lists the various paths involved.
|||The process identifier.|
|||This is a list of machines on which the service should run.|
|||These are the settings used to bring up the service (regardless of the machine). If changed, the service will be restarted.|
|||This is set by the service when it's up and running; the contents is specific to the implementation, but must be in JSON-format.|
Note that an ephemeral node is one that's created using the
zookeeper.EPHEMERAL flag and immediately removed when the creator
The machine id is a hardware UUIDs. On Linux this is the value returned by HAL for the
Each service is required to have a unique service name (a string).
To carry out tasks such as upgrades, coordinated script execution is needed.
Plugins can define tasks on a service level and make them available on the command-line:
$ pop run <service> <command> [args]
PythonEnvironment base class comes with a set of tasks that
help to set up the interpreter environment. These are available to all
services that derive from this base class.
Install packages. To make a Python package available in the instance environment:
$ pop run plone install lxml==2.3.5
Multiple packages can be listed, separate with space.
List packages. Return a list of installed Python libraries. For each package, print time of installation, source and version:
$ pop run plone packages
This section documents the development of pop (not with).
To run the automated test suite, you need a running ZooKeeper service and the nose test runner.
$ python setup.py nosetests
It's often useful to get a more verbose output:
$ python setup.py nosetests --verbosity=3
Note that a higher verbosity setting will result in a great amount of noise from the nose test runner itself.
Acknowledgements and Credits
The architecture and technical implementation of this software was inspired by Canonical's Juju cloud deployment tool, originally designed by Kapil Thangavelu. We deliberately use the same terms and conventions when possible (for example machines and services).
The author of this software:
Malthe Borch – email@example.com
Pop is available under the GPL.