fix: resolve SQLite embedding ValueError for list type (fixes #382)#397
Open
octo-patch wants to merge 1 commit intoNevaMind-AI:mainfrom
Open
fix: resolve SQLite embedding ValueError for list type (fixes #382)#397octo-patch wants to merge 1 commit intoNevaMind-AI:mainfrom
octo-patch wants to merge 1 commit intoNevaMind-AI:mainfrom
Conversation
…NevaMind-AI#382) The SQLite models inherited embedding: list[float] from parent classes but never overrode it with an SQLAlchemy-compatible column type. When SQLModel tried to create table columns, list had no matching SQLAlchemy type, raising ValueError. Replace the embedding_json + property pattern with a JSONEncodedList TypeDecorator that maps list[float] to TEXT via JSON serialization, matching how PostgreSQL models use Column(Vector()) to override the parent field.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #382
Problem
When using the SQLite backend, table creation fails with:
The SQLite model classes (
SQLiteMemoryItemModel,SQLiteResourceModel,SQLiteMemoryCategoryModel) inheritembedding: list[float] | Nonefrom parent Pydantic models. Whenbuild_sqlite_table_modelcreates table models withtable=True, SQLModel attempts to map all fields to SQL columns. Since SQLite has no native vector type andlisthas no default SQLAlchemy mapping, this raises aValueError.The PostgreSQL models avoid this by re-declaring
embeddingwithsa_column=Column(Vector(), ...), but the SQLite models used a separateembedding_jsonfield +@propertypattern that left the parentembeddingfield unmapped.Fix
JSONEncodedListSQLAlchemyTypeDecoratorthat serializeslist[float]to JSON text on write and deserializes back on readembeddingfield in each SQLite model withsa_column=Column(JSONEncodedList(), nullable=True), matching the PostgreSQL approachembedding_jsonfield and@propertygetter/setter boilerplaterow.embeddingdirectly (the TypeDecorator handles serialization transparently)Test
The fix can be verified by running
python tests/test_sqlite.py— table creation should no longer raiseValueError.