clj-zeppelin
A Clojure library for managing Zeppelin notebooks using the Zeppelin Notebook REST API.
Documentation
Installation:
- To install, add the following dependency to your project or build file:
[clj-zeppelin "0.1.0-SNAPSHOT"]
API Description
The operations defined in this library are as follows :
- create-note!
- list-notes
- delete-note!
- create-paragraph!
- run-paragraph-async
- run-paragraph-sync
- get-paragraph-status
- get-paragraph-info
- run-all-paragraphs
create-note!
This POST method creates a new note and returns the id of the newly created note.
- Function call :
(create-note! notebook-server-url payload)
- Input fields :
notebook-server-url - Zeppelin server url
payload - The payload is a map with (compulsory) key :name (which is the name of the notebook), and optional key :paragraphs, which is a list of paragraphs. Each paragraph is a map with keys :title and :text.
- Example :
(create-note! "http://[zeppelin-server]:[zeppelin-port]" {:name "Ocean Notebook"})
- Output :
2DTW93XQ9 is the id of the created note.
list-notes
This GET method returns a JSON array containing the name and id of all the available notes in the zeppelin server.
- Function call :
(list-notes notebook-server-url)
-
Input fields :
notebook-server-url - Zeppelin server url
-
Example :
(list-notes "http://[zeppelin-server]:[zeppelin-port]")
- Output :
{:status "OK", :message "", :body [{:name "Zeppelin Tutorial/Basic Features (Spark)", :id "2A94M5J1Z"}]}
delete-note!
This DELETE method deletes a note on the zeppelin server by the given note id and returns a JSON array containing status.
- Function call :
(delete-note! notebook-server-url note-id)
- Input fields :
notebook-server-url - Zeppelin server url
note-id - id of the notebook to be deleted
- Example :
(delete-note! "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9")
- Output :
{:status "OK", :message ""}
create-paragraph!
This POST method adds a new paragraph to an existing note and returns id of the newly created paragraph.
- Function call :
(create-paragraph! notebook-server-url note-id paragraph_data)
- Input fields :
notebook-server-url - Zeppelin server url
note-id - id of existing notebook in zeppelin
paragraph_data - content to be added to the paragraph
- Example :
(create-paragraph! "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" (-> {:title "intro" :text (10 + 10)}))
- Output :
20181029-070332_2137251371 is the id of the created paragraph.
run-paragraph-async
This POST method runs the paragraph asynchronously by given note and paragraph id and returns an OK message.
- Function call :
(run-paragraph-async notebook-server-url note-id paragraph-id)
- Input fields :
notebook-server-url - Zeppelin server url
note-id - id of the existing notebook in zeppelin
paragraph-id - id of the paragraph in the notebook
- Example :
(run-paragraph-async "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")
- Output :
OK
run-paragraph-sync
This POST method runs the paragraph synchronously by given note and paragraph id and returns a JSON array containing SUCCESS or ERROR status, depending on the outcome of paragraph execution.
- Function call :
(run-paragraph-sync notebook-server-url note-id paragraph-id)
- Input fields :
notebook-server-url - Zeppelin server url
note-id - id of the existing notebook in zeppelin
paragraph-id - id of the paragraph in the notebook
- Example :
(run-paragraph-sync "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")
- Output :
{:code "SUCCESS", :msg [{:type "TEXT", :data "\nres1: Int = 20\n"}]}
get-paragraph-status
This GET method gets the status of a single paragraph by the given note and paragraph id. The returned JSON array contains the paragraph id, paragraph status, paragraph finish date, paragraph start date.
- Function call :
(get-paragraph-status notebook-server-url note-id paragraph-id)
- Input fields :
notebook-server-url - Zeppelin server url
note-id - id of the existing notebook in zeppelin
paragraph-id - id of the paragraph in the notebook
- Example :
(get-paragraph-status "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")
- Output :
{:progress "100", :started "Mon Oct 29 07:20:06 UTC 2018", :finished "Mon Oct 29 07:20:07 UTC 2018", :id "20181029-070332_2137251371", :status "FINISHED"}
get-paragraph-info
This GET method retrieves an existing paragraph's information using the given id. The returned JSON array contains information about the paragraph.
- Function call :
(get-paragraph-info notebook-server-url note-id paragraph-id)
- Input fields :
notebook-server-url - Zeppelin server url
note-id - id of the existing notebook in zeppelin
paragraph-id - id of the paragraph in the notebook
- Example :
(get-paragraph-info "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")
- Output :
{:apps [], :dateFinished "Oct 29, 2018 7:20:07 AM", :jobName "paragraph_1540796612560_-766558253", :config {}, :dateUpdated "Oct 29, 2018 7:03:32 AM", :settings {:params {}, :forms {}}, :title "intro", :status "FINISHED", :id "20181029-070332_2137251371", :progressUpdateIntervalMs 500, :dateCreated "Oct 29, 2018 7:03:32 AM", :dateStarted "Oct 29, 2018 7:20:06 AM", :user "anonymous", :text "20", :results {:code "SUCCESS", :msg [{:type "TEXT", :data "\nres1: Int = 20\n"}]}}
run-all-paragraphs
This POST method runs all paragraphs in the specified note and returns a status message.
- Function call :
(run-all-paragraphs notebook-server-url note-id)
- Input fields :
notebook-server-url - Zeppelin server url
note-id - id of the existing notebook in zeppelin
- Example :
(run-all-paragraphs "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9")
- Output :
{:status "OK"}
Development
Testing:
- Install Docker
- Verify that Zeppelin Docker instance is available locally by running
docker run -p 8080:8080 apache/zeppelin:0.8.0 - Install Leiningen
- Run
lein testshould create the Zeppelin docker container and run the tests
License
Copyright © 2018 Dex
Distributed under the Apache 2.0 License
