Convert Lucene pending writes queue to use new generic PendingWritesQueue

[ohadzeliger]

Overview

The Lucene index keeps a persistent FDB-backed pending-writes queue that holds index operations (insert / delete) that arrive while a partition is locked during a merge.

This PR ports that queue onto the generalized core queue (…foundationdb.queue.PendingWritesQueue).

The new format wraps each entry in a small versioned envelope (PendingWritesQueueProto.PendingWriteItem: a version, a google.protobuf.Any payload, and an enqueue timestamp).
Readers understand both the new envelope format and the pre-existing legacy (serializer-wrapped) format.
Writers keep producing the legacy format until a property is flipped, then produce the new format. Nothing is rewritten in place.
The legacy Lucene PendingWriteQueue class is left untouched (legacy writer only) and becomes removable once every deployment writes the new format.

Design principles

Backwards compatibility with existing deployment - the system writes the old format be default so that the content can be parsed by older versions. Once a property is flipped, the new format will be used, which should be parsable by all instances once the deployment is complete.
Reading is always done using a fallback to the old format, so a mixed queue )old and new) can be drained correctly.
Key structure assumes incarnations are enabled to simplify the testing - this PR cannot be merged while queues with data without incarnations are still round.

version is the format discriminator — 0/absent = legacy, ≥1 = new; readers reject a newer-than-known version.
Opaque, forward-compatible payload — only the envelope is assumed; payload is a type-checked Any.
Legacy decoding is a read-side fallback; the decoder is used only when parsing as the current envelope fails or version == 0.

Rollout & rollback

Ship everywhere (reads both, writes legacy) → confirm deployed → flip the property (writes new). Rollback = unset the property; already-written new entries stay readable, new writes revert to legacy. No migration/rewrite; legacy entries age out as the queue drains.

What changed

Core queue read path (PendingWritesQueue, PendingWritesQueueEntry): reading distinguishes formats by the envelope version, exposes that version on the entry, and takes an optional legacy decoder on getQueueCursor(...) instead of baking it into the queue; toQueueEntry split into current-format and legacy-fallback paths.
Lucene wiring: FDBDirectoryWrapper owns/caches the queue and manages enqueue (routing by format), mirroring how it owns drain. FDBDirectory exposes the new-format queue factory, the legacy-writer factory, and the legacy decoder (PendingWritesQueueHelper.legacyDecoder(serializer)). Reads/drain/replay always use the generic queue with the legacy decoder supplied.
Format gate: new context property LUCENE_PENDING_WRITE_QUEUE_WRITE_NEW_FORMAT (default false).
Metrics: enqueue waits on a dedicated WAIT_LUCENE_PENDING_QUEUE_WRITE (the enqueue includes a capacity read).

Testing

• Core PendingWritesQueueLegacyFallbackTest — mixed new/legacy reads via the decoder; missing-decoder error; incarnation-less key rejected.

• PendingWriteQueueFormatCompatibilityTest — pure-protobuf: legacy parses as version 0, new not mistaken for legacy, mixed stream round-trips (× compression/encryption). (We can probably remove these by now)

• PendingWriteQueueIntegrationTest / PendingWriteQueueSizeIntegrationTest — parameterized over legacy vs new write format across the enqueue → replay → drain lifecycle and size limits.

Resolves #4353