Idempotent JSON serialisation/deserialisation

[rthewhite]

I currently have an entity which has a one to one relationship with another entity which is nullable.

When I retrieve the entity from the database while there is no foreign key value set, the result for that particular key/value is obviously None. When you then call to_json on the entity, this None value results in null in the resulting json. So far so good and understandable. (All though it would be nice to have an option to omit these null key/values completely, which I actually need in my case but that's a whole other thing).

What I would expect is that if I then do a new_from_json or update_from_json with this exact same json (with the null values), that this would work? (I have a REST API, client retrieves an entity, updates one property and passes back the entire object).

But this fails stating: 'NoneType' object is not iterable and only when I delete the entire key containing the null value it works. I think it would be nice that serialisation/deserialisation would be idempotent, meaning it should accept back what it outputs? Or I'm I overlooking something?

[insightindustry]

Thanks, @rthewhite - this looks like an edge case that our unit tests aren't catching: in other words, a bug.

In general, SQLAthanor should be idempotent and our unit tests generally are testing for that. Looks like this is a case is slipping through. I'll need to debug the issue and add a test case to make sure we're capturing it properly.

Would you be able to share a simplified version of the model that you're trying to serialize? Ideally the SQLAthanor serialization configuration and a couple of the columns from the model, in particular the offending key/value would be most helpful.

[rthewhite]

@insightindustry off course, hereby the 'pseudo' code.
Let me know if this is enough for you or whether you need more input :) More then happy to help, thanks for the awesome lib!

from uuid import uuid4
from datetime import datetime

from sqlathanor import Column, declarative_base
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.ext.declarative import declared_attr
from sqlalchemy import DateTime

Base = declarative_base()

class PrimaryMixin:
    @declared_attr
    def __tablename__(cls):
        return cls.__name__.lower()

    @declared_attr
    def uuid(self):
        return Column(
            UUID(as_uuid=True),
            primary_key=True,
            default=uuid4,
            unique=True,
            supports_json=(False, True),
        )

class DateMixin:
    @declared_attr
    def created(self):
        return Column(
            DateTime,
            default=datetime.utcnow(),
            supports_json=(False, True),
        )

    @declared_attr
    def lastUpdated(self):
        return Column(
            DateTime,
            default=datetime.utcnow(),
            onupdate=datetime.utcnow(),
            supports_json=(False, True),
        )


class Configuration(PrimaryMixin, Base):
    SomeField = Column(Integer, supports_json=True)
    SomeOtherField = Column(Float(), supports_json=True)
    # This has some more Integer and Float fields but not really relevant i guess

class Item(PrimaryMixin, DateMixin, Base):
    configurationUUID = Column(
        UUID(as_uuid=True),
        ForeignKey(Configuration.uuid),
    )
    configuration = relationship(
        Configuration,
        single_parent=True,
        supports_json=True,
        lazy="joined",
        cascade="all, delete-orphan",
    )

And the code to retrieve and update:

# Retrieval
query = db.session.query(Item)
result = query.filter(Item.uuid == 'foobar').one()
return result.to_json(max_nesting=3)

# Update
entity = db.session.query(Item).filter(Item.uuid == 'foobar').first()
entity.update_from_json(
    event["body"], drop_extra_keys=True, error_on_extra_keys=False
)