Yet Another Document Mapper (ODM) for MongoDB


Keywords
mongo, mongodb, mongodb-document, odm, orm, python
License
MIT
Install
pip install yadm==2.0.9

Documentation

Yet Another Document Mapper

https://travis-ci.org/zzzsochi/yadm.svg?branch=master https://coveralls.io/repos/github/zzzsochi/yadm/badge.svg?branch=master

It's small and simple ODM for use with MongoDB.

Requirements

YADM support MongoDB version 3.x only. MongoDB 2.x is not supported.

Minimal version of python — 3.6.

Quick start

Create the models

from datetime import datetime

import pymongo

from yadm import Database
from yadm import Document, EmbeddedDocument
from yadm import fields
from yadm.serialize import to_mongo


class User(Document):
    __collection__ = 'users'

    name = fields.StringField()
    email = fields.EmailField()


class PostLogItem(EmbeddedDocument):
    op = fields.StringField(choices=['created', 'comment_added'])
    at = fields.DatetimeField()
    data = fields.MongoMapField()


class Post(Document):
    __collection__ = 'posts'

    user = fields.ReferenceField(User)
    created_at = fields.DatetimeField(auto_now=True)
    title = fields.StringField()
    body = fields.StringField()
    log = fields.ListField(fields.EmbeddedDocumentField(PostLogItem))


class Comment(Document):
    __collection__ = 'comments'

    user = fields.ReferenceField(User)
    created_at = fields.DatetimeField(auto_now=True)
    post = fields.ReferenceField(Post)
    text = fields.StringField()

All documents creates from class Document. You can use multiple inheritance.

__collection__ magic attribute setups the collection name for documents of model.

Connect to database

client = pymongo.MongoClient('mongodb://localhost:27017')
db = Database(client, 'blog')

Database object is a wrapper about pymongo or motor Database.

Create documents

User

user = User(name='Bill', email='bill@galactic.hero')
db.insert_one(user)

Just insert document to database.

Post

post = Post()
post.user = user
post.title = 'Small post'
post.body = 'Bla-bla-bla...'
post.log = [PostLogItem(op='created', at=datetime.utcnow())]
db.insert_one(post)

You can fill documents as above.

Comment the post

comment = Comment()
comment.user = user
comment.post = post
comment.text = "RE: Bla-bla-bla..."
db.insert_one(comment)
db.update_one(post, push={
    'log': to_mongo(PostLogItem(op='comment_added',
                                at=comment.created_at,
                                data={
                                  'comment': comment.id,
                                  'user': comment.user.id,
                                }))
})

We add log item to post's log. This is very usefull case.

Queries

find

qs = db(Post).find({'title': {'$regex': '^S'}})
assert qs.count() > 0
  1. db(Post) creates the QuerySet object;
  2. find method get the raw-query and return new QuerySet object with updated criteria;
  3. count method make the query to database and return value.
for post in qs:
    assert post.title.startswith('S')

__iter__ method make the find-query and returns the generator of documents.

find_one

Get the first finded document.

post = db(Post).find_one({'user': user.id})

get_document

Get the document by id from primary.

user = db.get_document(User, user.id)

References

user = post.user

Get attribute with reference makes the query to referred collection. Warning: N+1 problem! We have a cache in QuerySet object and get one referred document only once for one queryset.

Lookups

comments = db(Comment).find({'post': post.id}).sort(('created_at', 1))
for comment in comments.lookup('user'):
    print(comment.user.name, comment.text)

This code create the aggregate query with $lookup statement for resolve the references.

Aggregations

agg = (db.aggregate(Comment)
       .match(user=user.id)
       .group(_id='post', count={'$sum': 1})
       .sort(count=-1))

for item in agg:
    print(item)

Or traditional MongoDB syntax:

agg = db.aggregate(Comment, pipeline=[
    {'match': {'user': user.id}},
    {'group': {'_id': 'post', 'count': {'$sum': 1}}},
    {'sort': {'count': -1}},
])

CHANGES

2.0.9 (2023-08-23)

  • Add comment methods for QuerySet and Aggregation to specify comment feature of MongoDB.

2.0.8 (2021-09-23)

  • Asyncio support for testing.

2.0.7 (2021-04-21)

  • Some bugfixes.

2.0.5 (2019-02-25)

  • Add Aggregation.hint method.

2.0.4 (2019-02-20)

  • Add Database.estimated_document_count method for quickly count documents in the collection.

2.0.1 (2018-11-04)

  • Add QuerySet.hint for specify index for query.

2.0.0 (2018-10-25)

  • A Big Rewrite document logic:
    • Document.__raw__ now contains only data from pymongo, without any AttributeNotSet or NotLoaded;
    • Document.__changed__ is removed: all changes reflects to Document.__cache__;
    • Document.__not_loaded__ frozenset of fields whitch not loaded by projection;
    • Document.__new_document__ flag is True for document's objects whitch created directly in your code;
    • Document.__log__ list-like container with log of document changes (unstable API at now);
    • Document.__data__ is removed as deprecated;
    • Now is not allow to set fields as classes;
    • Defaults is not lazy and creates with document instance;
  • Update for minimal versions of pymongo (3.7) and motor (2.0):
    • Add Database.bulk_write;
    • Add Database.insert_one, Database.insert_many and Database.delete_one;
    • Deprecate Database.insert, Database.remove;
    • Remove Database.bulk (without deprecation period, sorry);
    • Add QuerySet.count_documents;
    • Add QuerySet.update_one and QuerySet.update_many;
    • Add QuerySet.delete_one and QuerySet.delete_many;
    • Add QuerySet.find_one_and_update, QuerySet.find_one_and_replace and QuerySet.find_one_and_delete;
    • Deprecate QuerySet.count;
    • Deprecate QuerySet.update, QuerySet.remove and QuerySet.find_and_modify;
    • Remove deprecated QuerySet.with_id;
  • Simple interface for build lookups: QuerySet.lookup;
  • Remove bcc argument from MoneyField;
  • Add Decimal128Field.

1.5.0 (2017-12-31)

  • Experimental asyncio support;
  • Add ReferencesListField for lists of references.

1.4.15 (2017-12-27)

  • Add projection argument to Database.get_document and Database.reload;
  • Add Document.__default_projection__ attribute.

1.4.14 (2017-11-06)

  • Add EnumField for save enum.Enum;
  • Add EnumStateField for simple state machines based on enum.Enum.

1.4.13 (2017-10-31)

  • Add QuerySet.batch_size method for setup batch size for cursor;
  • Some minor fixes.

1.4.10 (2017-07-07)

  • ReferenceField.from_mongo try to get document from primary
    if not found by default.

1.4.9 (2017-07-06)

  • Add QuerySet.read_primary method for simple setup read_preference.Primary.

1.4.4 (2017-05-17)

  • Add TimedeltaField for stores durations;
  • Add SimpleEmbeddedDocumentField for simply create embedded documents.
class Doc(Document):
    embedded = SimpleEmbeddedDocumentField({
        'i': IntegerField(),
        's': StringField(),
    })

1.4.3 (2017-05-14)

  • Add StaticField for static data.

1.4.2 (2017-04-09)

  • Additional arguments (like write_concern) for write operations;
  • create_fake save the documents with write concern "majority" by default.

1.4.0 (2017-04-05)

  • Drop pymongo 2 support;
  • Additional options for databases and collections;
  • Add Database.get_document;
  • Add TypedEmbeddedDocumentField;
  • reload argument of Database.update_one must be keyword
    (may be backward incompotable).

1.3.1 (2017-02-21)

  • Change raw data for Money;

1.3.0 (2017-02-19)

  • Add currency support to Money:
    • Totaly rewrite Money type. Now it is not subclass of Decimal;
    • Add storage for currencies: yadm.fields.money.currency.DEFAULT_CURRENCY_STORAGE;

1.2.1 (2017-01-19)

  • Add QuerySet.find_in for $in queries with specified order;

1.2.0 (2016-12-27)

  • Drop MongoDB 2.X suport;
  • Objects for update and remove results;
  • Use Faker instead fake-factory.

1.1.4 (2016-08-20)

  • Add some features to Bulk:
    • Bulk.update_one(document, **kw): method for add update one document in bulk;
    • Bulk.find(query).update(**kw): update many documents by query;
    • Bulk.find(query).upsert().update(**kw): upsert document;
    • Bulk.find(query).remove(**kw): remove documents;

1.1.3 (2016-07-23)

  • Add QuerySet.ids method for get only documents id's from queryset;
  • Add Money.total_cents method and Money.from_cents classmethod;

1.1 (2016-04-26)

  • Add cacheing on queryset level and use it for ReferenceField;
  • Add mongo aggregation framework support;
  • Add read_preference setting;
  • Add exc argument to QuerySet.find_one for raise exception if not found;
  • Add multi argument to QuerySet.remove;
  • Deprecate QuerySet.with_id;
  • Refactoring.

1.0 (2015-11-14)

  • Change document structure. No more bad BaseDocument.__data__ attribute:
    • BaseDocument.__raw__: raw data from mongo;
    • BaseDocument.__cache__: cached objects, casted with fields;
    • BaseDocument.__changed__: changed objects.
  • Changes api for custom fields:
    • Not more need create field descriptors for every field;
    • prepare_value called only for setattr;
    • to_mongo called only for save objects to mongo;
    • from_mongo called only for load values from BaseDocument.__raw__;
    • Remove Field.default attribute. Use Field.get_default method;
    • Add Field.get_if_not_loaded and Field.get_if_attribute_not_set method;
    • By default raise NotLoadedError if field not loaded from projection;
  • Changes in ReferenceField:
    • Raise BrokenReference if link is bloken;
    • Raise NotBindingToDatabase if document not saved to database;
  • smart_null keyword for Field;
  • Fields in document must be instances (not classes!);
  • Remove ArrayContainer and ArrayContainerField;
  • Remove old MapIntKeysField and MapObjectIdKeysField. Use new MapCustomKeysField;
  • Add Database.update_one method for run simple update query with specified document;
  • Add QuerySet.distinct;
  • serialize.from_mongo now accept not_loaded sequence with filed names who must mark as not loaded, parent and name;
  • serialize.to_mongo do not call FieldDescriptor.__set__;
  • Fakers! Subsystem for generate test objects;
  • Tests now use pytest;
  • And more, and more...