Skip to content

Fix TypeError in save() when TopicMapper.mappings_ contains None (#2432)#2524

Open
lntutor wants to merge 1 commit into
MaartenGr:masterfrom
lntutor:fix/topicmapper-none-breaks-save-2432
Open

Fix TypeError in save() when TopicMapper.mappings_ contains None (#2432)#2524
lntutor wants to merge 1 commit into
MaartenGr:masterfrom
lntutor:fix/topicmapper-none-breaks-save-2432

Conversation

@lntutor

@lntutor lntutor commented Jul 26, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes #2432

Root cause

TopicMapper.add_new_topics pads the intermediate columns of a newly added row with None:

to_append = [key] + ([None] * (length - 2)) + [value]

As long as the mapping matrix has only two columns this is harmless (length - 2 == 0, so no None is inserted). However, once the matrix grows past two columns — which happens after repeated partial_fit calls — every new topic row gets one or more None placeholders. mappings_ is then no longer a homogeneous integer matrix.

Saving the model later calls _save_utils.save_topics, which converts the whole table with:

np.array(model.topic_mapper_.mappings_, dtype=int)

and this raises TypeError: int() argument must be ... not 'NoneType', exactly as reported in the issue's partial_fit + save(serialization="safetensors") reproduction.

Fix

A brand-new topic maps from itself in all prior states, so the intermediate columns should hold the topic id rather than None:

to_append = [key] * (length - 1) + [value]

This keeps mappings_ a homogeneous integer matrix so save() succeeds, and it is a no-op for the common two-column case ([key] * 1 + [value] == [key, value], identical to the previous output). On load, mappings_ is assigned back verbatim, so the rows round-trip unchanged.

Test

Adds test_topic_mapper_add_new_topics_keeps_integer_matrix in tests/test_bertopic.py. It builds a TopicMapper, widens the matrix past two columns to mimic repeated partial_fit states, calls add_new_topics, and asserts that np.array(mappings_, dtype=int) no longer raises, that pre-existing rows are unchanged, and that the new rows are homogeneous integers. The test is lightweight and does not train a model. It fails on the previous implementation (with the TypeError from the issue) and passes with this change.

Before submitting

TopicMapper.add_new_topics padded the intermediate columns of a new row
with None. Once the mapping matrix had more than two columns (which
happens after repeated partial_fit calls), mappings_ became a
non-homogeneous matrix and save() raised a TypeError when converting it
via np.array(mappings_, dtype=int).

A brand-new topic maps from itself in all prior states, so use the topic
id for the intermediate columns instead of None. This keeps mappings_ a
homogeneous integer matrix and leaves the two-column case unchanged.

Add a regression test that reproduces the >2-column scenario and asserts
the conversion no longer raises and existing rows round-trip unchanged.

Fixes MaartenGr#2432

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8b9dEaJtpvydzz4ttM7JH
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

model.save() fails with TypeError when topic_mapper_.mappings_ contains None

1 participant