We-Sci's seamless python toolset for tracking and managing research data


Keywords
biology, computational, provenance, we-sci, toolset, jupyter-notebook, open-science, precedence, provenance-analysis, provenance-model, python, research, sdk
License
MIT
Install
pip install wesci==0.6.2

Documentation

We-Sci Python Logger

Introduction

This repo contains instructions and examples of how to use We-Sci's Python Logger SDK to curate research done using python.

Current SDK Version

0.2.0

Using We-Sci's python logger:

Installation and Configuration

  1. Install We-Sci's pip package by running pip install wesci
  2. Follow the instruction you received in the email and create a a local config file (echo "<your_api_key> >> ~/.wesci.conf)

Initializing

import wesci

logger = wesci.Logger(
    script_file=__file__,
    log_file_prefix="./prefix"
)

### Your script starts here ###

The log_file_prefix parameter is used to specify the path and possibly the prefix of the local log file. e.g: (for script named script.py)

  1. If log_file_prefix=None --> log file will be stored at ./script_wesci_log
  2. If log_file_prefix=prefix --> log file will be stored at ./prefix_wesci_log
  3. If log_file_prefix=/log/prefix --> log file will be stored at /log/prefix_wesci_log

Adding Input/Output Parameters and Files

### Your script inputs ###
a1 = parameter_input
a2 = another_parameter_input

logger.add_input_params({'a1': a1,
                         'a2': a2})
logger.add_input_files({'input_csv': 'input.csv'})

### Your script calculations ###
# ...
# ...
# ...

b1 = some_ground_breaking_research_result
b2 = another_ground_breaking_research_result

logger.add_output_files({'output_csv': 'output.csv'})
logger.add_output_params({'b1': b1,
                          'b2': b2})

Adding Matplotlib figures

Coming soon...

Adding Images

Coming soon...

Logging

Just before the script exits, or at any point deemed important, insert the following code to log the data fed into the logger:

logger.log()

Note: Each call to logger.log() will record a new assay (experiment/test) - which will be displayed as a separate entity in your timeline on the We-Sci web app.

Jupyter Notebook support

Everything's basically the same, besides initialization:

logger = wesci.Logger(
    script_file='notebook_file.ipynb',  # Note that you need to specify the notebook's file
                                        # path explicitely!
    log_file_prefix="./prefix"          # the log file will be written to ./prefix_wesci_log.csv
)

Important: Only the code will be extracted from the notebook, with separation between cells, without images, data tables, etc.. Furthermore, the code is extracted from the last saved .ipynb file - not the one currently being evaluated, due to the architecture of Jupyter Notebook. We will be releasing a feature to dynamically capture the code in the near future.

Examples

You can see usage examples, both for a python script and python notebook at script_example.py and notebook_example.ipynb respectively.

Support

We currently maintain three channels for support:

  • For Python package related issues - you can open a git-issue.
  • For general issues, you can email us at support@we-sci.com.
  • For anything else, you can join the conversation on our Public-We-Sci-Users group on Slack.