add gossipsub v1.4 spec addressing reviewer feedback

pubsub/gossipsub/extensions/experimental/preamble-imreceiving.md

@@ -0,0 +1,112 @@
+# PREAMBLE and IMRECEIVING Extension
+
+| Lifecycle Stage | Maturity      | Status | Latest Revision |
+| --------------- | ------------- | ------ | --------------- |
+| 1A              | Working Draft | Active | r0, 2026-03-05  |
+
+Authors: @ufarooqstatus, @kaiserd
+
+Interest Group: TBD
+
+See the [lifecycle document][lifecycle-spec] for context about the maturity level
+and spec status.
+
+[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md
+
+## Overview
+
+This extension introduces two control messages — PREAMBLE and IMRECEIVING — to improve performance when handling large message transmissions in GossipSub. The extension is optional, fully compatible with GossipSub v1.2+, and involves minor modifications to peer behavior when receiving or forwarding large messages.
+
+The proposed modifications address the issue that the number of IWANT requests and duplicates increases significantly with message size. This happens because sending a large message can take considerable time, and during this time, the receiver is unaware of the IDs of the messages it is receiving.
+
+Under the current arrangement, if IHAVE announcements are received for a message that is already being received, the receiver may generate multiple IWANT requests, triggering unnecessary retransmissions of the same message. Higher message reception time also increases the probability of simultaneously receiving the same message from many peers.
+
+Sending a short control message, called PREAMBLE, before every large message transmission can allow receivers to immediately gain insights into the IDs and lengths of the messages they are receiving. This allows receivers to defer IWANT requests for messages they are already receiving, thereby reducing the number of unnecessary IWANT requests.
+
+At the same time, receivers may inform their mesh members about ongoing message reception. For this purpose, a new control message (IMRECEIVING) is introduced, indicating that the peer sending IMRECEIVING is currently receiving a message identified by the announced message ID, and sending duplicates might be unnecessary.
+
+On receiving an IMRECEIVING message, a peer should refrain from sending the message identified by the announced message ID. This can lead to a significant reduction in bandwidth utilization and message latency. The Safety Strategy below safeguards against malicious behavior.
+
+## Parameters
+
+This section lists the configuration parameters that clients need to agree on to avoid peer penalizations.
+
+| Parameter | Description |
+| --- | --- |
+| `peer_preamble_announcements` | The maximum number of PREAMBLE announcements for unfinished transfers per peer |
+| `max_iwant_requests` | The maximum number of simultaneous IWANT requests for a message |
+| `preamble_threshold` | The minimum message size (in bytes) required to enable the use of a message PREAMBLE |
+| `fallback_mode` | Message fetching strategy (`pull` or `push`) to use when a peer fails to deliver the message after sending a PREAMBLE. Default: `push` |
+
+## Message PREAMBLE
+
+### Basic scenario
+
+When a peer starts relaying a message that exceeds the `preamble_threshold` size, it should transmit a preceding control message called PREAMBLE.
+
+PREAMBLE serves as a commitment from the sender, indicating that the promised data message will follow without delay. A PREAMBLE must include the message ID and length, providing receivers with immediate access to critical information about the incoming message. The receiver must immediately process a PREAMBLE without waiting to download the entire message.
+
+On receiving a PREAMBLE that advertises a message ID not present in the seen cache, a receiver should add that message ID to the `ongoing_receives` list. The `ongoing_receives` list is crucial in limiting IWANT requests and simultaneous receptions of the same message from mesh peers.
+
+The receiver may use message length to conservatively estimate message download duration. If the message is successfully downloaded before the estimated download duration has elapsed, the message ID is removed from the `ongoing_receives` list and added to the seen cache.
+
+If the download takes longer than the estimated download time, the sender may be penalized through a behavioral penalty to discourage peers from intentionally delaying message transmission. See also Safety Strategy below.
+
+PREAMBLE is considered optional for both the sender and the receiver. This means that the sender can choose not to send the PREAMBLE, and the receiver can also opt to ignore it. Adding a PREAMBLE may increase control overhead for small messages. Therefore, it is preferable to use it only for messages that exceed the `preamble_threshold`.
+
+### Limiting IWANT Requests
+
+When a peer receives an IHAVE announcement for a message ID not present in the seen cache, the peer must also check the `ongoing_receives` list before making an IWANT request.
+
+If the message ID is found in the `ongoing_receives` list, the peer should postpone sending the IWANT request for a `defer_interval`. The `defer_interval` may be based on the estimated message download duration.
+
+If the message download completes before the `defer_interval` expires, the IWANT request for that message will not be generated. However, if the `defer_interval` elapses and the download has not completed, the peer can proceed to make the IWANT request for the missing message.
+
+The total number of outstanding IWANT requests for a single message must not exceed `max_iwant_requests`. Every peer must respond to incoming IWANT requests as long as the number of responses remains within the limits defined by gossipsub v1.1. Failing to do so should result in a behavioral penalty. This will discourage peers from intentionally not replying to IWANT requests.
+
+## IMRECEIVING Message
+
+The IMRECEIVING message serves a distinct purpose compared to the IDONTWANT message. An IDONTWANT can only be transmitted after receiving the entire message. In contrast, an IMRECEIVING should be transmitted immediately after receiving a PREAMBLE, indicating an ongoing large message reception. The IMRECEIVING message requests peers in the full message mesh to refrain from resending a large message that is already being received.
+
+When a peer receives a PREAMBLE indicating a message ID that is not present in the seen cache and `ongoing_receives` list, it should send IMRECEIVING messages to the nodes in its full message mesh. A peer MUST NOT send IMRECEIVING for the same messageID more than once. If a message transfer fails or the fallback timer expires, the peer falls back to v1.2 behavior and MUST NOT send another IMRECEIVING for that messageID. On receiving an IMRECEIVING from a peer, a node should refrain from sending the message indicated by the IMRECEIVING to that peer.
+
+A corresponding IDONTWANT from the peer that issued IMRECEIVING will indicate successful reception of the message. Otherwise, the peers in the full message mesh may proceed with a push-based or a pull-based operation depending upon the selected `fallback_mode`. See Safety Strategy below.
+
+## Safety Strategy
+
+A malicious peer can attempt to exploit this approach by sending a PREAMBLE but never completing (or deliberately delaying) the promised message transfer or by misrepresenting the message size, potentially hindering the message propagation across the network. The following describes one suggested defense mechanism. Sending and processing of PREAMBLE and IMRECEIVING messages is optional, and implementations may adopt a safety strategy that best fits their specific requirements.
+
+- A peer must accept and process a PREAMBLE only from mesh members and only once for any message ID.
+- A peer must limit the maximum number of PREAMBLE announcements for unfinished message transfers to `peer_preamble_announcements`.
+- A peer must extract the message ID and length from PREAMBLE to report in IMRECEIVING announcements.
+- A peer must ignore a received IMRECEIVING message if the reported length field does not correspond to the announced message ID.
+- If a peer identifies a misrepresented length field in a received PREAMBLE, it must penalize the sender for an invalid message.
+
+Upon receiving a PREAMBLE, a peer uses the length field to conservatively estimate message transfer duration, ensuring sufficient time allocation for message reception. If a message transfer is not complete during the estimated duration, the sender is penalized through a behavioral penalty, and message retrieval is performed based on a pull-based or a push-based strategy, as selected by the `fallback_mode`.
+
+In a pull-based strategy, the peer receiving the message uses an IWANT request to fetch the message. The peer can infer message availability at mesh members through received IDONTWANT announcements. In a push-based model, mesh members proactively send the message if they do not receive a corresponding IDONTWANT announcement within the fallback timer. Mesh members SHOULD use the `deferDurationMs` field from the IMRECEIVING message as their fallback timer, as the sender of IMRECEIVING is responsible for estimating its own download duration. If `deferDurationMs` is absent, mesh members may derive a conservative estimate based on the length field.
+
+Negative scoring helps prune non-conforming peers, whereas the fallback strategy helps recover from incomplete message transfers.
+
+## Protobuf
+
+```protobuf
+syntax = "proto2";
+
+message PreambleImreceivingExtension {
+  repeated ControlPreamble preamble = 1;
+  repeated ControlIMReceiving imreceiving = 2;
+}
+
+message ControlPreamble {
+  optional bytes  messageID     = 1;
+  optional uint64 messageLength = 2;
+}
+
+message ControlIMReceiving {
+  optional bytes  messageID       = 1;
+  optional uint64 messageLength   = 2;
+  optional uint32 deferDurationMs = 3;
+}
+```
+

pubsub/gossipsub/extensions/experimental/preamble-imreceiving.proto

@@ -0,0 +1,20 @@
+syntax = "proto2";
+
+// PREAMBLE and IMRECEIVING extension for improved large message handling
+// See: ./preamble-imreceiving.md
+
+message PreambleImreceivingExtension {
+  repeated ControlPreamble preamble = 1;
+  repeated ControlIMReceiving imreceiving = 2;
+}
+
+message ControlPreamble {
+  optional bytes  messageID     = 1;
+  optional uint64 messageLength = 2;
+}
+
+message ControlIMReceiving {
+  optional bytes  messageID       = 1;
+  optional uint64 messageLength   = 2;
+  optional uint32 deferDurationMs = 3;
+}

pubsub/gossipsub/extensions/extensions.proto

@@ -8,6 +8,10 @@ message ControlExtensions {
 
   optional bool testExtension = 6492434;
 
+  // PREAMBLE and IMRECEIVING extension for large message handling
+  // See: ./experimental/preamble-imreceiving.md
+  optional bool preambleImreceiving = 3847291;
+
 }
 
 message ControlMessage {
@@ -47,6 +51,10 @@ message RPC {
 
   optional TestExtension testExtension = 6492434;
 
+  // PREAMBLE and IMRECEIVING extension for large message handling
+  // See: ./experimental/preamble-imreceiving.proto
+  // optional PreambleImreceivingExtension preambleImreceiving = 3847291;
+
 }
 
 message PartialMessagesExtension {