github.com/guus-vanweelden/nds

A Go (golang) Google App Engine datastore package with strongly consistent caching.


Install
go get github.com/guus-vanweelden/nds

Documentation

nds

Build Status Coverage Status GoDoc

Package github.com/qedus/nds is a datastore API for the Google App Engine (GAE) Go Runtime Environment that uses memcache to cache all datastore requests. It is compatible with both Classic and Managed VM products. This package guarantees strong cache consistency when using nds.Get* and nds.Put*, meaning you will never get data from a stale cache.

Exposed parts of this API are the same as the official one distributed by Google (google.golang.org/appengine/datastore). However, underneath github.com/qedus/nds uses a caching stategy similar to the GAE Python NDB API. In fact the caching strategy used here even fixes one or two of the Python NDB caching consistency bugs.

You can find the API documentation at http://godoc.org/github.com/qedus/nds.

One other benefit is that the standard datastore.GetMulti, datastore.PutMulti and datastore.DeleteMulti functions only allow you to work with a maximum of 1000, 500 and 500 entities per call respectively. The nds.GetMulti, nds.PutMulti and nds.DeleteMulti functions in this package allow you to work with as many entities as you need (within timeout limits) by concurrently calling the appropriate datastore function until your request is fulfilled.

How To Use

You can use this package in exactly the same way you would use google.golang.org/appengine/datastore. However, it is important that you use nds.Get*, nds.Put*, nds.Delete* and nds.RunInTransaction entirely within your code. Do not mix use of those functions with the google.golang.org/appengine/datastore equivalents as you will be liable to get stale datastore entities from github.com/qedus/nds.

Ultimately all you need to do is find/replace the following in your codebase:

  • datastore.Get -> nds.Get
  • datastore.Put -> nds.Put
  • datastore.Delete -> nds.Delete
  • datastore.RunInTransaction -> nds.RunInTransaction