Serialize Django Data with MOM

June 4, 2021

Model Object Mapper for Django

django  python  serialization

Model Object Mapper is a Django command-line utility that lets you serialize, create and update database entries.

Model Object Mapper, or MOM for short, is a Django Management Utility for statically creating and updating database entries. It supports relational fields and can work with Django apps without requiring any modification. It is free and open-source, and licensed under MIT License.

Introduction

MOM improves your workflow when dealing with personal projects or private datasets that don't require end-user-friendly forms to insert or update. It offers an easy-to-use serialization mechanism that syncs your database every time you run it.

Usage

This section explains some of the most useful features of MOM. You can find the extensive documentation and quickstart guide here:

Modeling After Models

To have MOM recognize our Django models, we first describe how we will use them in a file called Main MOM File. In that file, we tell MOM which files point to which models and how we will handle relational fields.

To illustrate that, let's assume that we have the following model:

# File: home/models.py

from django.db import models

class Post(models.Model):
    slug = models.SlugField(primary_key=True, )
    title = models.CharField(max_length=100, )
    date = models.DateTimeField()
    content = models.TextField()

Now, in the Main MOM File, you can do:

# File: mom_data/mom.yaml

mom:
    map:
        post:
            model: home.models.Post
            lookupField: slug

Here the map key inside the mom key defines the post key that tells MOM how to handle a home.models.Post model. Following that, MOM finds the files whose names start with post, and continue with the value it will assign to the slug field of home.models.Post, and finally end with the MOM file format. When we bring all three together, we get something like this: post.slug-field-value.mom.yaml.

Inside the Objects

Since we have created a Main MOM File, we can now focus on objects (or serialization files). In these files, we represent database tables and fields, which will then enable us to create new rows and update existing ones as the files change.

Now, let's assume that we are creating the post.slug-field-value.mom.yaml file that we have talked about:

# File: mom_data/post.slug-field-value.mom.yaml

field:
    title: My Awesome Post
    date: 2021-02-23 10:25:00+3
    content: This is an awesome post.

The value of the slug field is slug-field-value since it is coming from the file name.

Now you can run:

$ ./manage.py mom

As a result, MOM will find this file and use the slug field and its value to query for an existing row. If there is one, MOM will compare its fields for changes. If there isn't one, MOM will create a new one on the database.

One useful feature you can use here is the additional options that you can enable by adding a space after the field name. For instance, if you are working with a markdown file and want to use it as the value for the content field, you can do this: 

# File: mom_data/post.slug-field-value.mom.yaml

field:
    # ...    
    content file: content.md

and inside the content.md file:

[comment]: # File: mom_data/content.md

This is an awesome post.

This way, you can work with different file formats outside of YAML files.

MOM has many features like this that you can use right now, and they are explained in the documentation thoroughly. Relational fields may be a good reason to check it out.

Development

When I began working on this, I had a lot of experience with Django, but not with Python, specifically with creating Python packages. If you have been there, you probably know there is a lack of standardization in that regard. If Cargo is the official package manager, build system, a tool for testing, documentation, and publication for Rust, for Python, well, you can say it is pip, setuptools, build, unittest, venv, and twine. Finding these out was easy but not necessarily straightforward.

Getting Started

I didn't want to complicate things, so I started small and created MOM as a management utility for my website. There, I ensured myself that it is working and I could move it into a standalone package.

After that, I continued developing the code, but I could not set up testing early in the development, so I decided to edit the installed venv package and put the changes back in the standalone code.

Dependencies

I am still not sure if setuptools can install dependencies. pyproject.toml can install build-time dependencies, but it removes them after the build, so it is useless for me unless it allows to set runtime dependencies and install them.

Testing

unittest does the job, and the official Django documentation recommends it as well.

I created another Django app in the root folder to run the tests, which seemed to be the only good option if we want to avoid bloating the django_mom package.

After the test cases were ready, setting up the CI runners was easy. I used drone locally first, then set up a GitHub Workflow.

Publication

The recommended way is to not go in a rush and push to pypi.org. Instead, publishers should either use test.pypi.org or their instance to see everything is alright.

Also, just so you know, after publishing a package, you can delete a release, but you cannot re-upload it with the same version and architecture.

Conclusion

MOM can improve your workflow with the flexibility it provides and can be helpful in many different ways.

Adopting MVVM