[pickle] Dump using the highest available protocol#261
Conversation
| # I didn't found a good way to avoid pickling/unpickling if | ||
| # key is smaller than chunksize, because in case or <werkzeug.requests> | ||
| # getting the length consume the data iterator. | ||
| serialized = pickle.dumps(value, 2) |
There was a problem hiding this comment.
I'm uncertain why this was hardcoded as 2.
|
While I am generally for this PR I am not sure if this might break some applications? |
| content_type = "application/json" | ||
| try: | ||
| value = json.dumps(value) | ||
| value = json.dumps(value, pickle.HIGHEST_PROTOCOL) |
There was a problem hiding this comment.
Apologies. I got a tad carried away.
|
@sh4nks regarding,
per here:
My take is that by using |
3 participants
This PR ensures that all backends use the same protocol (the highest available) for pickling the payload to cache. Some backends currently use the default (4 for Python 3.8) whereas some use 2 and others use the highest protocol available (maximum of 5).
Per the
pickledocumentation,it seems there is merit in utilizing the highest available protocol for consistency and increased performance.
Note that it seems the protocol is encoded within the dumped object, i.e., one does not need to re-specify the protocol when loading via
pickle.loads(...), thus this should be a non-breaking change.