clfpy – Python library for accessing platform services of the SemWES platform
This is a simple Python library that offers access to infrastructure services in SemWES (formerly CloudFlow) and its derivatives. The library aims to make things like interaction with GSS (file upload, download, etc.) as easy and hassle-free as possible.
Command-line interface (CLI)
With version 1.0.0 of the library, a command-line interface was added for fully
interactive use. Simply install clfpy with pip and then execute
from any terminal. For more detailed documentation, check the CLI's readme
A big thank you goes to Leonardo Cosma from CETMA who initiated the creation of the CLI and also provided some of the code.
2020-12-21: Version 1.3.1
- (BUGFIX) Fixed Barbora support
2020-12-21: Version 1.3.0
- (MINOR) Added support for Barbora cluster
2020-10-12: Version 1.2.0
- (MINOR) Added method to services client to pull a Docker image
2019-04-24: Version 1.1.3
- (PATCH) Fix bad hardcoded interpreter in clfpy_cli script
2019-03-29: Version 1.1.2
- (PATCH) Re-brand CloudFlow to SemWES
2019-01-17: Version 1.1.1
- (PATCH) Fix CLI tab completion for service names
2019-01-16: Version 1.1.0
- (MINOR) Added missing methods to CLI auth client
- (MINOR) Improved tab completion in the CLI
2019-01-07: Version 1.0.1
- (PATCH) Fixed broken input for Python 2.7
2019-01-07: Version 1.0.0
- (MAJOR) Added full command-line interface for the library
- (MINOR) Added missing list_users method to
- (PATCH) Added better exception handling to
2018-12-14: Version 0.10.1
- (PATH) Added exception handling to
2018-12-11: Version 0.10.0
- (MINOR) Added
clfpy.ExtraParametersclass for parsing extraParameters strings
- (PATCH) Fixed Python 2.x problem for
2018-12-10: Version 0.9.0
- (MINOR) Added convenience function
clfpy.ServicesClient.read_env_file()for reading environment-definition files when defining services.
2018-12-07: Version 0.8.1
clfpy.ServicesClient: Added health description in case of unhealthy targets.
2018-12-07: Version 0.8.0
- (MINOR) New client
clfpy.ServicesClientfor automatic service deployment in CloudFlow
- (PATCH) Fixed HPC endpoint in
2018-11-27: Version 0.7.0
- (MINOR) File upload and download now shows a status bar and the average speed by default.
2018-10-31: Version 0.6.2
- (PATCH) Bugfix: GSS test script fails with Python 2.7
2018-10-31: Version 0.6.1
- (PATCH) Bugfix: Uploading an empty file fails
Currently, the following clients are available:
clfpy.AuthClient: Client for the authentication manager
clfpy.AuthUsersClient: Client for the users interface of the authentication manager
clfpy.AuthProjectsClient: Client for the projects interface of the authentication manager
clfpy.GssClient: Client for accessing the generic storage services (GSS)
clfpy.HpcImagesClient: Client for registering Singularity images with the CloudFlow HPC client
clfpy.WfmClient: Client for interacting with the workflow manager (does not yet expose the full WFM functionality)
clfpy.ServicesClient: Client for automatic service deployment. Requires Docker to be installed!
clfpy.ExtraParameters for parsing extraParameters inputs in
Requires Python 2.7 or Python 3.x.
clfpy.ServicesClient, Docker must be installed as well and the
Docker CLI must be available under the
clfpy can be installed from the Python Package Index using pip:
pip install clfpy
How to use
Have a look at the
clfpy/tests/ folder to find examples on how to use the
How to upload a new version to PyPi
- Make changes to the code
- Run test scripts with Python 2.7 and Python 3.x.
- Update the changelog and choose a new version number using semantic versioning.
- Update the version number in
- Remove all files from
./buildto avoid attempting to overwrite existing files when uploading.
python setup.py sdist bdist_wheeland check the resulting files in
twine upload dist/*to upload to PyPi. Note that this step requires a correctly configured