papi
Low-Boilerplate RESTful APIs
Introduction
papi is a library that allows you to build powerful RESTful web services on top of plain WSGI by writing backends as simple and semantic classes, and then feeding them to its equally simple WSGI wrapper function.
Features
- Proper RESTful semantics over HTTP(S): GET, PUT, POST, DELETE map to retrieve / list resources, create, update, delete
- Automatic routing
- Automatic HATEOAS decoration (adds links to parent, self, and children, on every JSON response)
- Semi-automatic content type negotiation: JSON is handled transparently, other content types are easy to support in your backend code
- Automatic translations of failures to HTTP error responses; uses the 4xx range of status codes correctly
- Runs on any compliant WSGI host, making it suitable for deployment under a wide range of web servers and protocols
- Method override: fake unsupported HTTP methods through GET parameters or headers
Installing
Installing with pip:
pip install papi
You probably want to do this in a virtualenv.
Conceptual Model
Papi's concept of a RESTful API is that of a tree-shaped data structure, consisting of "leaf" nodes called "documents" and "branch" nodes called "collections". Both are modelled as resources, and a resource can act as a document, as a collection, or both. Documents have a body (potentially available in multiple flavors, matching different MIME content types); collections have child resources, and the library code maps this resource tree onto a URL path structure. As RESTfulness has it, HTTP methods indicate the kind of operation on this tree, and the HATEOAS philosophy is applied by tagging documents and collections with metadata when possible.
Usage
Defining A Resource
To implement a working RESTful API, you need to define a root resource. Resources can act as documents (having a body), collections (having child resources), or hybrids (having both a body and child resources). For the root resource, you almost certainly want a collection-style resource, otherwise your API will only ever contain one document.
Note that Resource
is not a base class, it's just an implicit
interface. Papi resolves method calls through duck typing, there is
no need to inherit or formally implement anything, just add the
methods you need, and that's it. Adding other methods is of course no
problem at all.
The relevant methods for a resource are:
def get_structured_body(self, digest=False)
def get_typed_body(self, mime_pattern)
Get the payload data for the resource itself; implementing these methods makes the resource a document.
get_typed_body
is always tried first; it should return a pair of
(mime_type, body)
to indicate that a body is available that matches
the mime_pattern
, or None
to tell Papi that this MIME type
cannot be satisfied.
For some "special" MIME types (currently only text/json
and
application/json
), the get_structured_body
method is tried when
get_typed_body
fails; this method is supposed to return a native
Python data structure. Currently, the only requirement is that the
returned data must be JSON-encodable, but in the future, other types may
be supported (e.g. XML, plain text, HTML, ...), so it's best to stick
with "vanilla" data structures that directly correspond to JSON types:
dict
, list
, tuple
, int
, float
, bool
, str
and
None
are all safe to use, others might not. The digest
argument
indicates whether the full body should be returned, or a "digest"
version that contains only the essential properties. digest
will be
True
when called on a child resource in a collection listing
context, False
when the resource is requested directly.
One thing to keep in mind with these two methods is that documents derived from ``get_typed_body`` are never parsed, and no metadata is ever added. This means that if you want to have Papi add HATEOAS links and a list of child resources to the response, you must implementget_structured_body
, and if you also implementget_typed_body
, it must returnNone
for at least the JSON content types (and, in the future, any content type you want to have tagged with metadata).
def get_children(self, offset=0, count=10, filters=None, order=None)
def get_child(self, name)
These methods need to be implemented for resources that act as
collections. get_children
returns a list of (name, resource)
pairs, and can take the following keyword arguments to alter its behavior:
-
filters
: a list ofFilter
objects. AFilter
object has three properties:operator
,value
, andpropname
, wherepropname
indicates which property of the document to compare,operator
indicates how to compare (currently only"equals"
is used), andvalue
is a (string) value that the property is compared against. -
-
order
: a list of(descending, order-key)
pairs, from most-significant - to least-significant. If
descending
isTrue
, the result must be ordered in descending order.order-key
is specific to the resource, no further interpretation is performed by Papi.
-
-
offset
: the number of items to skip from the beginning of the list. Works like Python'sx[offset:]
construct, or theOFFSET
part in an SQLLIMIT
clause. -
count
: the number of items to return, starting at theoffset
if provided. Works like Python'sx[:count]
construct, or theCOUNT
part in an SQLLIMIT
clause. -
page
: whencount
is specified, you can provide a page number instead of anoffset
. Page numbers are 1-based, and each page containscount
entries, sopage=2, count=10
retrieves items 10 through 19.
It is recommended to implement get_children
with additional *args
and
**kwargs
arguments, such that future Papi versions can add additional
arguments without breaking compatibility.
get_child
gets a single child resource; the name
parameter,
throughout Papi's Python API, refers to a resource's primary key. We
call it "name", because ideally, it should be a somewhat descriptive,
meaningful natural identifier for the object it represents, which, when
possible, is more in line with the RESTful philosophy, and makes for
naturally beautiful URIs.
http://example.org/api/fruit/apples/granny_smith
is a much nicer URI
than http://example.org/api/5d75e3/35b0bd/d68c481bb1f4
.
def create(self, input, content_type=None)
def store(self, input, name, content_type=None)
def delete(self, name)
These methods can optionally be implemented to turn a readonly resource
into a writeable collection. Note that all write operations are
defined on the parent resource, even though at the HTTP level, some are
exposed on the resource itself - for example, POST /root/child1
maps
to the resource named "child1"
under the parent resource "root"
,
but the method that gets called is the store
method of the root
resource. This is for two reasons: one, the child resource to store may
not exist yet (this is the case for PUT
requests), and two, the
resource itself does not know its own name, nor does it need to.
Some notes on these methods:
The input
argument will contain a file-like object, which means
you can use the usual read()
etc. methods on it to extract the
body. Parsing is your own responsibility, Papi does not do this for
you. Particularly, there is no write equivalent to the
get_structured_body
method; however, processing JSON documents is
usually a simple matter of calling json.loads
.
The difference between create
and store
is that create
must generate a name for the received document, and return a
name, body
tuple (where body
is a digest that describes the
document that has been created, in a JSON-encodable data structure
according to the same rules as get_structured_body
); multiple
calls to create
should create multiple distinct documents, and
return distinct names. Conceptually, create
always creates a
new document. By contrast, store
takes a document name as an
argument, so it does not generate one itself, and multiple calls with
the same name will overwrite one another. While store
may also
create new documents (if the name
does not exist yet), it should
overwrite (update) documents when the name already exists.
Serving A Resource
Serving a resource is simple; the serve_resource
function can be
used to turn a valid resource into a WSGI application, like this:
application = serve_resource(root_resource)
And from there, it's a matter of feeding that function to a WSGI server (see the WSGI documentation for details).
Give It A Spin
The included example application (example/app.py
) implements a
simple in-memory database that supports plain-text payloads for
documents; all the resources in it are read/write document/collection
hybrids, which means that data can be added at any point in the tree.
Assuming that this application runs in a WSGI server on localhost:5000,
we can try a few requests (we'll use cURL for these examples):
> curl 'http://localhost:5000/' # Fetch the root resource
{"_parent": {"href": "/"}, "_self": {"href": "/"}, "_items": [{"_parent":
{"href": "/"}, "_self": {"href": "/things"}, "_name": "things"}]}
That's not very readable, but we can use the pretty
parameter to
pretty-print JSON output:
> curl 'http://localhost:5000/?pretty=1'
{
"_parent": {
"href": "/"
},
"_self": {
"href": "/"
},
"_items": [
{
"_parent": {
"href": "/"
},
"_self": {
"href": "/things"
},
"_name": "things"
}
]
}
This tells us a few things:
- The URI for this resource (
_self
) is/
- The URI for this resource's parent (
_parent
) is also '/' (this is actually a misfeature currently; the root node should not actually report a parent) - The resource contains child resources (
_items
) - To be specific, it contains one child resource, named
things
, with a URI of/things
.
As you can see, this HATEOAS metadata makes the API fully discoverable; the resource tells us its own location within the API, as well as those of its parent and children.
Let's look at the child resource "things":
> curl 'http://localhost:5000/things/?pretty=1'
{
"_parent": {
"href": "/"
},
"_self": {
"href": "/things"
},
"_items": [
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/apple"
},
"_value": "I am an apple. Eat me.",
"_name": "apple"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/banana"
},
"_value": "I'll bend either way for you.",
"_name": "banana"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/nut"
},
"_value": "I'm nuts!",
"_name": "nut"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/onion"
},
"_value": "Hurt me, and I will make you cry.",
"_name": "onion"
}
],
"_name": "things"
}
Oh joy! What a bunch of things! And they're still fully HATEOAS-discoverable, so let's see what happens when we try to fetch an onion:
> curl 'http://localhost:5000/things/onion/?pretty=1'
Hurt me, and I will make you cry.
That's weird. No JSON. Why is that? Right, content negotiation. Our
example resource supports text/plain
as well as JSON; curl, by
default, specifies that it accepts */*
, that is, anything, and
because Papi prefers "typed" bodies over "structured" bodies, the first
type that matches (which happens to be text/plain
) is what we get.
If we were serving, say, images through our API, this would be exactly
the desired behavior. We can still request JSON though, we just have to
override the Accept
header:
> curl 'http://localhost:5000/things/onion/?pretty=1' -H 'Accept: text/json'
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/onion"
},
"_value": "Hurt me, and I will make you cry.",
"_name": "onion"
}
All is well!
So far, we have only requested things that existed. Of course requesting
something that doesn't exist yields a 404 error; we'll use cURL's -i
option to show HTTP headers:
> curl 'http://localhost:5000/things/nope/?pretty=1' -i
HTTP/1.1 404 Not Found
Content-type: text/plain;charset=utf8
Not Found
That makes sense.
What happens if we request a content type that the resource doesn't support?
> curl 'http://localhost:5000/things/onion/?pretty=1' -i -H 'Accept: img/png'
HTTP/1.1 406 Not Acceptable
Content-type: text/plain;charset=utf8
Not Acceptable
It does the right thing.
So far we've only been reading from the API; let's try writing
things. According to standard RESTful procedures, we can create new
documents by using the HTTP PUT
method:
> curl 'http://localhost:5000/things/potato' -XPUT -i -H 'Content-Type: text/plain'
HTTP/1.1 200 OK
Content-type: application/json
{"_parent": {"href": "/things"}, "_self": {"href": "/things/potato"}, "_value": "Slice me, dice me, fry me"}
The status code 200
indicates that the document was indeed created,
and fetching the _self
URI confirms this:
> curl 'http://localhost:5000/things/potato/?pretty=1'
Slice me, dice me, fry me
And of course, this new document supports JSON as well:
> curl 'http://localhost:5000/things/potato/?pretty=1' -H 'Accept: text/json'
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/potato"
},
"_value": "Slice me, dice me, fry me",
"_name": "potato"
}
Note that if you want to access the API from a web browser, it will
almost certainly not support any HTTP methods other than GET
and
POST
(plus a few that we don't care much about here, such as
HEAD
and OPTIONS
); PUT
and DELETE
, in particular, will
not work. Because of this, Papi has a method override feature: if you
add a _method
parameter to the query string, or a
X-Method-Override
header to the request, the value of that will
override the actual request method. So the following curl requests would
all produce the same behavior:
> curl 'http://localhost:5000/things/potato' -XPUT -i -H 'Content-Type: text/plain'
> curl 'http://localhost:5000/things/potato?_method=PUT' -XPOST -i -H 'Content-Type: text/plain'
> curl 'http://localhost:5000/things/potato' -XPOST -i -H 'X-Method-Override: PUT' -H 'Content-Type: text/plain'
An alternative way of creating new documents is using the HTTP method
POST
on the parent resource, leaving the responsibility of
generating a suitable unique name for the new document to the parent
resource. This is what that looks like:
> curl 'http://localhost:5000/things?pretty=1' -XPOST -i -H 'Content-Type: text/plain' -d'Carrot on a stick'
HTTP/1.1 200 OK
Content-type: application/json
{"_parent": {"href": "/things"}, "_self": {"href": "/things/carrot"}, "_value": "Carrot on a stick"}
Our example resource is configured to generate names based on the first
word of the input, so that's what we get: "carrot"
.
Other than the PUT
method, however, POST
will always create a
new document, rather than overwrite an existing one, so if we POST
the same thing again, the API is required to either deny the request
with a Conflict
response, or create a new document with a different
unique name. Our example application opts for the second solution:
> curl 'http://localhost:5000/things?pretty=1' -XPOST -i -H 'Content-Type: text/plain' -d'Carrot on a stick'
HTTP/1.1 200 OK
Content-type: application/json
{"_parent": {"href": "/things"}, "_self": {"href": "/things/BL6yCijd8x4Mwzcf-carrot"}, "_value": "Carrot on a stick"}
As you can see, the name is disambiguated by prepending a random token.
Listing the /things
resource shows that two documents have actually
been created:
> curl 'http://localhost:5000/things?pretty=1' -H 'Accept: text/json'
{
"_parent": {
"href": "/"
},
"_self": {
"href": "/things"
},
"_items": [
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/BL6yCijd8x4Mwzcf-carrot"
},
"_value": "Carrot on a stick",
"_name": "BL6yCijd8x4Mwzcf-carrot"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/apple"
},
"_value": "I am an apple. Eat me.",
"_name": "apple"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/banana"
},
"_value": "I'll bend either way for you.",
"_name": "banana"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/carrot"
},
"_value": "Carrot on a stick",
"_name": "carrot"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/nut"
},
"_value": "I'm nuts!",
"_name": "nut"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/onion"
},
"_value": "Hurt me, and I will make you cry.",
"_name": "onion"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/potato"
},
"_value": "Slice me, dice me, fry me",
"_name": "potato"
}
],
"_name": "things"
}
And of course our example application also supports deleting items,
using the DELETE
method:
> curl 'http://localhost:5000/things/potato/?pretty=1' -i -XDELETE
HTTP/1.1 204 No Content
Content-type: text/plain
Note the use of the 204 No Content
status line; since we've deleted
a resource, there is no meaningful content to return, all we get is an
empty success response. And to confirm that the potato has indeed been
deleted:
> curl 'http://localhost:5000/things?pretty=1' -H 'Accept: text/json'
{
"_parent": {
"href": "/"
},
"_self": {
"href": "/things"
},
"_items": [
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/BL6yCijd8x4Mwzcf-carrot"
},
"_value": "Carrot on a stick",
"_name": "BL6yCijd8x4Mwzcf-carrot"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/apple"
},
"_value": "I am an apple. Eat me.",
"_name": "apple"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/banana"
},
"_value": "I'll bend either way for you.",
"_name": "banana"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/carrot"
},
"_value": "Carrot on a stick",
"_name": "carrot"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/nut"
},
"_value": "I'm nuts!",
"_name": "nut"
},
{
"_parent": {
"href": "/things"
},
"_self": {
"href": "/things/onion"
},
"_value": "Hurt me, and I will make you cry.",
"_name": "onion"
}
],
"_name": "things"
}