execute vs executemany

[paigewilliams]

Excuse me for a potentially dumb question, but I was wondering why you chose to use psycopg.execute vs psycopg.executemany? I have a project that is using this library, and another that is using psycopg in a non-beam environment. In my non-beam environment I have seen performance improvements by using executemany over execute so I was curious about the functions usage here. Thanks for entertaining my question!

[medzin]

Hi, it looks like a simple oversight :) I will update the lib next week. I remember that executemany had bad implementation a long time ago in psycopg2, but I'm using psycopg3 here, which has the optimal one. Thanks for catching this!

[paigewilliams]

Cool! Looking forward to checking out the change!

[medzin]

Ok, so there will be a problem with executemany usage - the code will no longer know which row caused the exception if it happens. It would make it impossible to have retry logic like the one here: https://github.com/medzin/beam-postgres/blob/main/beam_postgres/io/postgres.py#L228-L235