Analysis kit for large-scale structure datasets, the massively parallel way


Keywords
python, clustering, mpi, parallel-computing, cosmology, astrophysics, large-scale-structure, data-analysis, mpi4py
License
GPL-3.0
Install
pip install nbodykit==0.3.15

Documentation

nbodykit: a massively parallel large-scale structure toolkit


nbodykit is an open source project and Python package providing a set of algorithms useful in the analysis of cosmological datasets from N-body simulations and large-scale structure surveys.

Driven by the optimism regarding the abundance and availability of large-scale computing resources in the future, the development of nbodykit distinguishes itself from other similar software packages (i.e., nbodyshop, pynbody, yt, xi) by focusing on :

  • a unified treatment of simulation and observational datasets by insulating algorithms from data containers
  • reducing wall-clock time by scaling to thousands of cores
  • deployment and availability on large, super-computing facilities
  • an interactive user interface that performs as well in a Jupyter notebook as on super-computing machines

All algorithms are parallel and run with Message Passing Interface (MPI).

Build Status

We perform integrated tests of the code, including all built-in algorithms, in a miniconda environment for Python 2.7, 3.5, and 3.6.

Build Status Test Coverage Conda PyPi

Documentation

The official documentation is hosted on ReadTheDocs at http://nbodykit.readthedocs.org/.

Cookbook Recipes

Users can dive right into an interactive cookbook of example recipes using binder. We've compiled a set of Jupyter notebooks to help users learn nbodykit by example — just click the launch button below to get started!

binder

Users can also view a static version of the cookbook recipes in the documentation.

Installation

We recommend using the Anaconda distribution of Python. To obtain the dependencies and install a package on OSX or Linux, use

$ conda install -c bccp nbodykit

We are considering support for Windows, but this depends on the status of mpi4py.

Using nbodykit on NERSC

On the Cori and Edison machines at NERSC, we maintain a nightly conda build of the latest stable release of nbodykit. See the documentation for using nbodykit on NERSC for more details.

Bumping to a new version

  1. git pull - confirm that the master branch is up-to-date
  2. Edit Changelog (CHANGES.rst) - Make sure to include all issues which have arisen since the last version. (git add ... -> git commit -m "Update Changelog" -> git push)
  3. Edit version.py -> git push ("bump version to ...")
  4. Go to https://travis-ci.org/bccp/nbodykit and make sure it merged without any problems.
  5. Go to bccp/conda-channel-bccp repo and do "Restart build"
  6. git tag 0.3.? -> git push --tags
  7. bump to a development version (0.3.?dev0)

Acknowledgement

The work on nbodykit is supported by the Berkeley Center for Cosmological Physics, University of California Berkeley, by the National Energy Research Scientific Computing Center of the Lawrence Berkeley National Lab via the allocation m3035, and by users and contributors.