Datadots API for Python
STATUS: What you currently see here is WIP, do not use in production!
This is the Datadots API for Python, allowing you to access and manipulate dots against any Dotmesh cluster. Note that only Python 3 is supported.
Install
You must have Python 3 installed, this is the only version supported. Then do:
$ pip3 install datadots-api==0.1.4Use
Some usage examples below.
Create a dot
from dotmesh.client import DotmeshClient
d = DotmeshClient(cluster_url=cluster_url, username=user, api_key=api_key)
dot = dmclient.createDot(dotname="test")Create a dot in a namespace
from dotmesh.client import DotmeshClient
d = DotmeshClient(cluster_url=cluster_url, username=user, api_key=api_key)
dot = dmclient.createDot(dotname="test", ns="mynamespace")Look up a dot by name
from dotmesh.client import DotmeshClient
d = DotmeshClient(cluster_url=cluster_url, username=user, api_key=api_key)
print(d.getDot(dotname="test"))Commit to a branch and show the log
from dotmesh.client import DotmeshClient
d = DotmeshClient(cluster_url=cluster_url, username=user, api_key=api_key)
dot = dmclient.createDot(dotname="test")
branch = dot.getBranch("master")
branch.commit("just a test commit")
print(branch.log())Commit to a branch with some extra metadata
from dotmesh.client import DotmeshClient
d = DotmeshClient(cluster_url=cluster_url, username=user, api_key=api_key)
dot = dmclient.createDot(dotname="test")
branch = dot.getBranch("master")
branch.commit("a commit with some extra metadata", {"fruit": "apples", "color": "red"})
print(branch.log())Create new branch based on latest master
from dotmesh.client import DotmeshClient
d = DotmeshClient(cluster_url=cluster_url, username=user, api_key=api_key)
dot = dmclient.createDot(dotname="test")
branch = dot.getBranch("master")
branch.commit("just a test commit")
mybranch = branch.createBranch("mybranch", "master")
mybranch.commit("the first commit in my branch")
print(mybranch.log())End-to-end example
You can see this API in action, for a local setup with Docker running and after you've created a cluster using dm cluster init, by running the following example:
$ DOTMESH_APIKEY=$(cat ~/.dotmesh/config | jq -r .Remotes.local.ApiKey)
$ DOTMESH_APIKEY=$DOTMESH_APIKEY python3 example.py
== Create dot:
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Create", "params": {"Namespace": "admin", "Name": "test"}, "id": 1}
<-- {"jsonrpc":"2.0","result":true,"id":1} (200 OK)
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Lookup", "params": {"Namespace": "admin", "Name": "test", "Branch": ""}, "id": 2}
<-- {"jsonrpc":"2.0","result":"32965cef-4310-4291-6f36-c86106655d10","id":2} (200 OK)
ID=32965cef-4310-4291-6f36-c86106655d10, name=test
== Get master branch:
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Lookup", "params": {"Namespace": "admin", "Name": "test", "Branch": ""}, "id": 3}
<-- {"jsonrpc":"2.0","result":"32965cef-4310-4291-6f36-c86106655d10","id":3} (200 OK)
master
== Do some commit and show log:
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Commit", "params": {"Namespace": "admin", "Name": "test", "Branch": "", "Message": "just a test commit"}, "id": 4}
<-- {"jsonrpc":"2.0","result":true,"id":4} (200 OK)
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Commit", "params": {"Namespace": "admin", "Name": "test", "Branch": "", "Message": "and another commit, who would have thought"}, "id": 5}
<-- {"jsonrpc":"2.0","result":true,"id":5} (200 OK)
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Commits", "params": {"Namespace": "admin", "Name": "test", "Branch": ""}, "id": 6}
<-- {"jsonrpc":"2.0","result":[{"Id":"1007db99-11cb-4bd7-747e-1d121bb9b11a","Metadata":{"author":"admin","message":"just a test commit","timestamp":"1521214312285694689"}},{"Id":"67bb6184-430b-46d7-4337-b858ee85eb1c","Metadata":{"author":"admin","message":"and another commit, who would have thought","timestamp":"1521214312418429042"}}],"id":6} (200 OK)
1521214312285694689: just a test commit
1521214312418429042: and another commit, who would have thought
== Create new branch, commit and show log:
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Commits", "params": {"Namespace": "admin", "Name": "test", "Branch": ""}, "id": 7}
<-- {"jsonrpc":"2.0","result":[{"Id":"1007db99-11cb-4bd7-747e-1d121bb9b11a","Metadata":{"author":"admin","message":"just a test commit","timestamp":"1521214312285694689"}},{"Id":"67bb6184-430b-46d7-4337-b858ee85eb1c","Metadata":{"author":"admin","message":"and another commit, who would have thought","timestamp":"1521214312418429042"}}],"id":7} (200 OK)
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Branch", "params": {"Namespace": "admin", "Name": "test", "SourceBranch": "master", "NewBranchName": "mybranch", "SourceCommitId": "67bb6184-430b-46d7-4337-b858ee85eb1c"}, "id": 8}
<-- {"jsonrpc":"2.0","result":true,"id":8} (200 OK)
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Commit", "params": {"Namespace": "admin", "Name": "test", "Branch": "mybranch", "Message": "the first commit in my branch"}, "id": 9}
<-- {"jsonrpc":"2.0","result":true,"id":9} (200 OK)
--> {"jsonrpc": "2.0", "method": "DotmeshRPC.Commits", "params": {"Namespace": "admin", "Name": "test", "Branch": "mybranch"}, "id": 10}
<-- {"jsonrpc":"2.0","result":[{"Id":"565b04ab-abae-476a-75bd-203bce2a56b2","Metadata":{"author":"admin","message":"the first commit in my branch","timestamp":"1521214312967840418"}}],"id":10} (200 OK)
1521214312967840418: the first commit in my branchNote that the default behavior of the example above is to leave the dot in place, that is, if you do a dm dot show test you should see both the master branch and mybranch with the commits. If you want to automatically clean up, execute it as follows: python3 example.py cleanup:
$ dm list
Current remote: local (use 'dm remote -v' to list and 'dm remote switch' to switch)
DOT BRANCH SERVER CONTAINERS SIZE COMMITS DIRTY
* test master 995e5c821c38e7f5 1.00 kiB 1 -
$ dm dot show test
Dot admin/test:
Master branch ID: 7f4a4d67-5bc7-47aa-7829-31d5a02ff70f
Dot is current.
Dot size: 1.00 kiB (all clean)
Branches:
* master
mybranch
$ dm checkout mybranch && dm log
commit 565b04ab-abae-476a-75bd-203bce2a56b2
Author: admin
Date: 1521214312967840418
the first commit in my branchTest
Locally, you can test the Dotmesh JSON-RPC API like shown in the following. It assumes you've got http installed (or alternatively you can use curl) as well as that Docker is running (note: tested on v18.03):
$ dm cluster init
$ DOTMESH_CLUSTERURL=http://localhost:6969/rpc
$ DOTMESH_APIKEY=$(cat ~/.dotmesh/config | jq -r .Remotes.local.ApiKey)
$ http -a admin:$DOTMESH_APIKEY POST $DOTMESH_CLUSTERURL < test/ping.json
HTTP/1.1 200 OK
Content-Length: 57
Content-Type: application/json; charset=utf-8
Date: Fri, 16 Mar 2018 07:06:44 GMT
X-Content-Type-Options: nosniff
{
"id": 6129484611666146000,
"jsonrpc": "2.0",
"result": true
}Release
To release this project to pypi, check the version (git tag and find the highest one, or check setup.py) then run bump2version --new-version <new-version> <major|minor|patch> && git push --tags
