Duratom

Where

What

A durable atom type for Clojure. Duratom implements the same interfaces as the core Clojure atom (IAtom, IRef, IDeref). In order to provide durability duratom will persist its state to some durable-backend on each mutation. The built-in backends are:

1. A file on the local file-system
2. A postgres DB table row
3. An AWS-S3 bucket key

Note: Several ideas taken/adapted/combined from enduro & durable-atom

Main difference between duratom & enduro is that an enduro atom is not a drop-in replacement for regular clojure atoms. In particular:

1. it doesn't implement all the same interfaces as regular clojure atoms. As a result it comes with its own swap! & reset! implementations.
2. it requires the watches/validators to be provided in atoms upon construction.

Main difference between duratom & durable-atom is that a durable-atom atom doesn't have a second level of polymorphism to accommodate for switching storage backends. It assumes that a file-backed atom is always what you want. Moreover, it uses slurp & spit for reading/writing to the disk, which, in practice, puts a limit on how big data-structures you can fit in a String (depending on your hardware & JVM configuration of course). Finally, it uses locking which is problematic on some JVMs (e.g. certain IBM JVM versions). duratom uses the java.util.concurrent.locks.Lock interface instead.

Usage

The public API consists of a single constructor function (duratom.core/duratom). Once you have constructed a duratom object, you can use it just like a regular atom, with the slight addition that when you're done with it, you can call duratom.core/destroy on it to clear the durable backend (e.g. delete the file/table).

Subsequent mutating operations are prohibited (only derefing will work).

Example

;; backed by file
(duratom :local-file
         :file-path "/home/..."
         :init {:x 1 :y 2})

;; backed by postgres-db
(duratom :postgres-db
         :db-config "any db-spec understood by clojure.java.jdbc"
         :table-name "my_table"
         :row-id 0
         :init {:x 1 :y 2})

;; backed by S3
(duratom :aws-s3
         :credentials "as understood by amazonica"
         :bucket "my_bucket"
         :key "0"
         :init {:x 1 :y 2})

The initial-value is ignored, unless the underlying persistent storage is found to be empty. If you prefer passing arguments positionally, you can use the file-atom, postgres-atom & s3-atom equivalents.

Custom :read & :write

By default duratom stores plain EDN data (via pr-str). If that's good enough for your use too, you can skip this section. Otherwise, observe the following examples of utilising the :rw keyword for specifying nippy as the encoder/decoder of that EDN data:

;; nippy encoded bytes on the filesystem 
(duratom :local-file
         :file-path "/home/..."
         :init {:x 1 :y 2}
         :rw {:read nippy/thaw-from-file
              :write nippy/freeze-to-file})

;; nippy encoded bytes on PostgresDB
(duratom :postgres-db
         :db-config "any db-spec understood by clojure.java.jdbc"
         :table-name "my_table"
         :row-id 0
         :init {:x 1 :y 2}
         :rw {:read nippy/thaw
              :write nippy/freeze
              :column-type :bytea})
          
;;nippy encoded bytes on S3 
(duratom :aws-s3
         :credentials "as understood by amazonica"
         :bucket "my_bucket"
         :key "0"
         :init {:x 1 :y 2}
         :rw {:read #(with-open [dis (DataInputStream. %)]
                       (nippy/thaw-from-in! dis))
              :write nippy/freeze})          

Requirements

Java/Clojure >= 1.7

Optional Requirements

• clojure.java.jdbc >= 0.6.0
• amazonica

License

Copyright © 2016 Dimitrios Piliouras

Distributed under the Eclipse Public License, the same as Clojure.